Research Academy
My account Customers About Blog Careers Legal Contact Resellers

Burp Suite User Forum

Create new post

Crawling a site with cookies

Raf | Last updated: Nov 14, 2022 01:49PM UTC

Hello, I am trying to crawl (right click on the site root in Target, select Scan, select Crawl in Scan type) a web site that uses cookies for session tracking. I get "Paused task due to: Could not connect to any seed URLs" error and the scan is paused. I have looked at the Logger and I see that Burp sends a request to the site root page and gets a redirect to the login page (which is located at another domain). It does not send any cookies with the request. After trying it 3 times it gives up. I have tried enabling the session rule "Use cookies from Cookie jar" for Target and Scanner. I have looked at the session tracer and saw a message: "Vetoing rule: Use cookies from Burp's cookie jar", which explains why cookies aren't sent. I have tried using Burp Suite Navigator Recorder plugin. I recorded my login sequence and added it in Crawl configuration in Application Login -> Use recorded login sequences . This had no interesting effects. The Logger does not record any requests related to login and I still see "Vetoing rule: Use cookies from Burp's cookie jar", so the crawl fails. I also tried entering the username and password in Application Login -> Use login credentials. It had the same result. What is the proper way of crawling the site that uses session cookies, using those session cookies? Is there a way to disable "smart" cookie management in crawler and just have it use the cookies from the cookie jar? I have used Burp Suite Pro v2022.11 and v2022.9.6. The behaviour seems to be the same for both.

Michelle, PortSwigger Agent | Last updated: Nov 15, 2022 08:41AM UTC

Thanks for getting in touch. To help us look into this for you can you email some screenshots of the steps you're taking to set up your crawl to support@portswigger.net, please? When you configured the crawl to use a recorded login sequence, did the logs report any errors when trying to log in, or did they report that locations had been found after logging in?

You must be an existing, logged-in customer to reply to a thread. Please email us for additional support.