Research Academy
My account Customers About Blog Careers Legal Contact Resellers

Burp Suite User Forum

Create new post

make live passive crawl more powerful !

chenyun | Last updated: Sep 16, 2021 10:59AM UTC

I really like burpsuite, thank you for providing such a powerful tool! I recently found that the function of tools scope in live passive is not powerful enough. In 2021.8.2 version , There are only three options Proxy,Repeater,Intruder,not scanner.Imagine a scenario like this. when i use passively scan this branch in the sitemap ,I hope that some of the newly discovered item can be integrated into the sitemap. Please add this feature if it is convenient. Thanks again!

Michelle, PortSwigger Agent | Last updated: Sep 16, 2021 02:24PM UTC

Thanks for your message. To help us fully understand your requirements can you tell us a bit more about your use case, please? Do you make use of the scan tasks to perform an automated crawl of your sites, or do you generally perform a manual crawl and browse around the site yourself to find content and the areas you want to investigate? Which items are missing from the sitemap?

chenyun | Last updated: Sep 17, 2021 04:22AM UTC

Thank you for your reply! When you open a new project, two live tasks are opened by default, one is live passive crawl and the other is live passive audit. When you browse around site,burp can analyze links in http responses and add them to the sitemap and display them in gray, burp not sure if these links are alive. Some are black, proving that burp gets the response to these links. When I select a branch in the sitemap, using the a function `passively scan this branch`,you can see scanner traffic in the logger. In this case, I want live passive crawl to analyze the scanner traffic and detect which links are live and which are not, so that it can update that branch in the sitemap. Perhaps you could add `scanner` to the `live passive crawl options`.`Scan details`.`Tools Scope` Thanks~

Michelle, PortSwigger Agent | Last updated: Sep 17, 2021 10:05AM UTC

chenyun | Last updated: Sep 19, 2021 12:33PM UTC

Thanks for your reply ! :） The two materials you mentioned I have understood in detail. Thanks again ~ Sometimes I need to crawl just one branch.In this case I prefer to select `passively scan this branch` in the sitemap rather than start a `New scan`.`Crawl`. Although the latter can be done, it is a bit of a hassle...... So if `live passive crawl options`.`Scan details`.`Tools Scope` has `Scanner`. `live passive crawl` will auto update the sitemap when i select `passively scan this branch`. This will make everything easy. On the other hand. When performing `Do passive scan` on a url in a sitemap, burp will update the sitemap if the response status code is 200. However, if `passively scan this branch` is executed on a branch, burp will not update the sitemap, regardless of the result. So if add `scanner` to the `live passive crawl options`.`Scan details`.`Tools Scope` . Can solve this problem. Thanks again and looking forward to your reply!

Michelle, PortSwigger Agent | Last updated: Sep 20, 2021 10:57AM UTC

At the stage where you want to crawl just one branch, are you ideally just wanting to automatically discover what else may be linked from that page or are you wanting to identify vulnerabilities within that branch? If there was an option to start the crawl-only scan task with default options from the right-click menu (instead of having to choose Scan and go through the scan task dialog), would this help?

chenyun | Last updated: Sep 26, 2021 02:23AM UTC

I'm sorry for only replying now I didn't describe my question clearly at the beginning, and in talking with you I got a clearer picture of what I wanted to say. What I'm trying to describe is a bug When you select a url or branch in sitemap and right-click to do passive scan or passively scan this branch, you can see the requests triggered by the scanner in the logger. burp will update the corresponding entry in sitemap based on the traffic triggered by the scanner . This is more likely when the response status code is 200, but sometimes it still does not update. However, when the traffic is triggered by proxy, the sitemap will be updated regardless of the response status code. So, I propose a solution. add `scanner` to the `live passive crawl options`. `Scan details`. `Tools Scope` . Of course, your mention of If there was an option to start the crawl-only scan task with default options from the right-click menu is also a solution, but then the burp doesn't take full advantage of the traffic triggered by passive scan-triggered traffic. This is why I said my problem is more of a bug. Looking forward to your reply!

Michelle, PortSwigger Agent | Last updated: Sep 27, 2021 10:57AM UTC

Hi Thanks for the update. When you select a URL or branch in sitemap and right-click to do 'Passive scan' or 'Passively scan this branch' Burp will use the details from the requests and responses it has already in the history rather than making new requests. The only new requests that will be made by default will be ones that are related to JavaScript Analysis, the audit phase will not be looking for new locations to scan, so no new locations would be added to the site map unless an automated crawl, further manual crawling was done or the Repeater/Intruder tools send more requests. If you wanted to audit the results from both a manual crawl and an automated crawl you could perform both crawl stages and then use the right-click option on the Site map to start the audit, this is where an option to right-click and automatically crawl a branch form within the site map could be a useful addition for you.

chenyun | Last updated: Sep 28, 2021 08:03AM UTC

Thank you for your patience in explaining these details to me. It's true that I got some things mixed up. I think the improved features you mentioned are indeed a good solution. Please add this feature if it is simple to implement. :) Thanks again~

Michelle, PortSwigger Agent | Last updated: Sep 30, 2021 09:47AM UTC

We've added your vote to the feature request to add an option for a crawl-only scan on the right-click menu. We don't have any timescales for it but we've linked this thread so we can post back here with any updates.

chenyun | Last updated: Oct 05, 2021 02:46AM UTC

thanks a lot ~

You must be an existing, logged-in customer to reply to a thread. Please email us for additional support.