Research Academy
My account Customers About Blog Careers Legal Contact Resellers

Burp Suite User Forum

Create new post

Checkbox for unauthenticated/authenticated crawl/audit

Simko, | Last updated: Nov 10, 2020 10:45AM UTC

Hi, could you please introduce some checkbox, whether the Crawl+Audit should be authenticated, unauthenticated, or both? I have a huge scope (thousands of JSP files), I set login, and after 24h it's still only making unauthenticated crawl, whereas I wouldn't want that in the first place in this case. Having 2 simple check-boxes would be amazing to have, so that I could only select one of them, or both, depending the scope. Thanks

Ben, PortSwigger Agent | Last updated: Nov 11, 2020 09:11AM UTC

Hi Andrej, This is fairly intrinsic behavior - the Burp Scanner is designed to first perform an unauthenticated crawl in order to map the content and discover any login and self registration functions within the web application. After this phase is complete, it then moves on to map content beyond the login function itself. In essence, the unauthenticated crawl provides a baseline of content that is then expanded upon during the authenticated crawling stage. You can certainly mimic the behavior of a completely unauthenticated scan by not supplying any login credentials and disabling the two options under Login Functions within the crawl configuration (and the default behavior, when login credentials are supplied, is to combine both an unauthenticated and authenticated scan).

Simko, | Last updated: Nov 23, 2020 10:13PM UTC

Thanks for reply. Even though Burp Suite is designed this way, with the new macro recorder; if I choose to record macro, is there still a need for unauthenticated scan? I mean if I switch off the auto registration, there shouldn't be any need for it, right? I ask because the website I'm testing really has thousands of JSP files, so even the crawling phase for unauth it almost timing out the scan, and I'd like to be able to turn off this unauth crawl phase. I get that I can adjust depth of crawl, etc. but I'm interested if, in the future, it could be possible to simply not perform unauthenticated crawl when login macro is supplied, and self-registration disabled. Thanks:)

Ben, PortSwigger Agent | Last updated: Nov 25, 2020 04:26PM UTC

Hi Andrej, As part of the crawl and audit process, Burp records the path walked to get to a particular location. It then uses that information during the audit phase in order to determine the shortest path required to reach that request from the initial starting point. If, during the crawl phase, Burp discovers that the session handling mechanism is more complex and requires Burp to obtain a fresh token, perhaps by issuing the preceding request first, then it has the ability to re-walk this path during the audit phase in order to do this. If Burp can issue requests directly, because it is accessing a location where no session tokens are required, then it will do this. By performing an unauthenticated crawl first, Burp has this baseline to work from and can distinguish between areas of the site that are accessible without a session and those that are behind login functions (and might require more complex steps to be reached during the audit phase).

You must be an existing, logged-in customer to reply to a thread. Please email us for additional support.