Research Academy
My account Customers About Blog Careers Legal Contact Resellers
The Burp Suite User Forum was discontinued on the 1st November 2024.

Burp Suite User Forum

For support requests, go to the Support Center. To discuss with other Burp users, head to our Discord page.

SUPPORT CENTER DISCORD

Force spider engine to wait for page to load (Automated spider)

Sebastien | Last updated: May 07, 2018 10:14AM UTC

Hello, I was testing an intensive application this week and noticed that the spider tool wasn't finding a lot of the content on the site. The spidering was done through a scheduled task, so there was no manual browsing involved. I believe the cause of this is that the site is very JavaScript intensive, and it takes about 5 seconds before all the content is actually loaded on the site. (The site itself loads pretty fast, it's just the JavaScript content that's slow) Therefore I was wondering if it was possible to force the spider engine to wait x amount of seconds before it starts looking for other URL's. I already tried setting the delay between spider requests to 20 seconds, but this still leaves me with the problem that the spider engine only lists the URL's that are available when the page loads (without any of the JavaScript content), and just delays the requests between these found URL's. There also doesn't seem to be any extensions that provide this sort of functionality. Thanks

PortSwigger Agent | Last updated: May 08, 2018 08:52AM UTC