Research Academy
My account Customers About Blog Careers Legal Contact Resellers
The Burp Suite User Forum was discontinued on the 1st November 2024.

Burp Suite User Forum

For support requests, go to the Support Center. To discuss with other Burp users, head to our Discord page.

SUPPORT CENTER DISCORD

Best way to crawl (traditional web sites as well as heavy javascript and AJAX using websites)

vivek | Last updated: Oct 05, 2020 07:57AM UTC

Hi, I want to know how I can use burp to best crawl and auto submit forms on a website as an unauthenticated user for both traditional web sites as well as heavy javascript and AJAX using websites. I tried a test case yesterday where I gave burp a page to crawl (https://www.w3schools.com/tags/tryit.asp?filename=tryhtml_form_method) and it was unable to go to the iframe and submit the simple GET based form. Can you please tell me how it can be done and how can burp be made to do the best crawling and auto form submission in general.

Hannah, PortSwigger Agent | Last updated: Oct 06, 2020 08:36AM UTC

Hi Due to this example being in an iframe, for this case, you will need to disable browser-driven scanning to crawl properly and submit the form. Iframe support for browser-driven scanning is an improvement on our roadmap. Generally speaking though, for crawling more modern websites we would recommend ensuring that browser-driven scanning is enabled. You can find this option in your scan configuration, under "Crawling > Miscellaneous > Use embedded browser for crawl and audit".

vivek | Last updated: Oct 06, 2020 01:50PM UTC

Thanks for the clarification :)

vivek | Last updated: Oct 07, 2020 02:12PM UTC

Hi there, I disabled the browser-driven scanning but the crawler was still not able to submit the form in the iframe. Any suggestions? And in your knowledge is there any crawler better for this job which I can use with burp or stand alone? Best Regards, Vivek

Hannah, PortSwigger Agent | Last updated: Oct 08, 2020 07:28AM UTC