Research Academy
My account Customers About Blog Careers Legal Contact Resellers
The Burp Suite User Forum was discontinued on the 1st November 2024.

Burp Suite User Forum

For support requests, go to the Support Center. To discuss with other Burp users, head to our Discord page.

SUPPORT CENTER DISCORD

Burp Pro 2020.2.1 Crawler Not Finding Resource

Pentester | Last updated: Mar 27, 2020 11:13PM UTC

Hello, I'm trying to troubleshoot an issue with Burp Suite Professional's crawler and "Live passive crawl from Proxy" with Burp Suite Professional 2020.2.1. The following request is not being discovered and added to the sitemap by 2020.2.1 Burp Pro's crawler, but IS being discovered by the spider and added to the sitemap in v1.7.37 and I'm trying to figure out why. On Burp Pro 2020.2.1 tried crawling with "Crawl strategy - most complete" configuration, and the live passive crawling configuration has it set to Types of item to add: Links / Form submissions and URLs to add: Everything GET /%2bCSCOE%2b/message.html?mc=2 HTTP/1.1 Host: <REDACTED> Accept-Encoding: gzip, deflate Accept: */* Accept-Language: en User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0) Connection: close Referer: https://<REDACTED>/%2bCSCOU%2b/ Cookie: webvpnlogin=1; webvpnLang=en The filter in the sitemap is similar across both Burp Pro versions, and it is running on the same Kali Linux operating system. Any ideas why this particular request would be ignored / not added to the sitemap by Burp Pro 2020.2.1 and discovered by the previous version 1.7.37? Assistance is appreciated.

Pentester | Last updated: Mar 27, 2020 11:34PM UTC

To give some further context, this is from a standard Cisco SSL VPN login page. It seems odd that the spider would find it and the crawler would not, especially if it's more advanced and effective as stated.

Liam, PortSwigger Agent | Last updated: Mar 30, 2020 09:17AM UTC