Research Academy
My account Customers About Blog Careers Legal Contact Resellers

Burp Suite User Forum

Create new post

On huge targets, sitemap stops working reliably

Aleksandr | Last updated: Jul 31, 2023 01:24PM UTC

I don't know the exact conditions but often some paths from the passive crawl are missing from sitemap. They're present in the proxy tab but never added to the sitemap. The problem isn't new and I know people who struggle with this as well.

Dominyque, PortSwigger Agent | Last updated: Jul 31, 2023 02:05PM UTC

Hi Aleksandr On the Dashboard, do you have the 'live passive crawl from Proxy' enabled? If yes, how many items is it saying are being added to the site map? Additionally, can you double-check that you haven't enabled to drop all out-of-scope requests? Please feel free to email support@portswigger.net with any screenshots you think will be helpful!

Aleksandr | Last updated: Aug 03, 2023 07:30AM UTC

Hi, I have 150k requests and all of them are processed within the live passive crawl. Also, I didn't set up scope so it shouldn't be a reason.

Aleksandr | Last updated: Aug 03, 2023 07:33AM UTC

The paths are unique as well. Some of them even colored in gray because they were present in js files.

Dominyque, PortSwigger Agent | Last updated: Aug 03, 2023 11:43AM UTC

Hi Aleksandr Can you please email support@portswigger.net with example screenshots of certain paths not displaying in the site map so we can investigate it further?

You must be an existing, logged-in customer to reply to a thread. Please email us for additional support.