Research Academy
My account Customers About Blog Careers Legal Contact Resellers

Burp Suite User Forum

Create new post

Unable to scan all Urls of site map at once

Khizra | Last updated: Jan 16, 2020 11:55AM UTC

Hi, I am using Burp Suite Professional 2.1.7. I browsed all urls of a website they were displayed in target site map then i added the host to scope. After clicking on the main url of website, if i scan it through audit and crawl options, only one url is scanned. How do i make burp scan the complete host (including all browsed urls) in a single scan? We are able to add all the urls manually in scan but how can we automatically scan all the urls at once?

Hannah, PortSwigger Agent | Last updated: Jan 16, 2020 01:22PM UTC

When you add your item to scope, are you using the parent folder (for example https://portswigger.net) or a URL that is a child of the parent folder (for example https://portswigger.net/bappstore)? If you have the parent as the included URL in your target scope (or the URL to scan when configuring the scan), it will target all of the pages within that folder group. You can find some information on starting scans here: - https://portswigger.net/burp/documentation/desktop/scanning - https://portswigger.net/burp/documentation/desktop/scanning/scan-launcher Our full documentation for Professional can be found here: - https://portswigger.net/burp/documentation/desktop

Hardik | Last updated: Jun 16, 2020 09:10AM UTC

Hi, I did the same. added the item to sitemap(for example https://portswigger.net) added this site to scope. After clicking on the main url of website in sitemap tab, if i scan it through audit and crawl options, only one url is scanned i.e https://portswigger.net with path /

Hannah, PortSwigger Agent | Last updated: Jun 16, 2020 09:26AM UTC

Hi Hardik Can you tell me the version of Burp you are using? Our most up-to-date version is currently v2020.5. When you launch a scan from the dashboard and include the parent as your URL to scan (eg. https://portswigger.net), does your scan complete the crawl phase? Are there any errors shown in your dashboard?

Hardik | Last updated: Jun 16, 2020 12:18PM UTC

Hi, Yes I am using v2020.5 Yes I completed the scan with crawl and audit option. There are 2 errors: Unrecognized command line arguement --headless.mode could not start browser But I dont think this has anything to do with my problem.

Hardik | Last updated: Jun 16, 2020 12:42PM UTC

I have rectified these errors also still getting same. After clicking on the main url of website in sitemap, if i scan it through audit and crawl options, only one url is scanned. While in Contents tab(just right of sitemap),all browsed urls are there.( when i click on main url of website in sitemap)

Hannah, PortSwigger Agent | Last updated: Jun 16, 2020 01:22PM UTC

Hi Hardik If you go to your dashboard, can you tell me the number of locations crawled and the number of requests made for your task?

Hardik | Last updated: Jun 16, 2020 02:24PM UTC

12 locations and 5000 requests

Hardik | Last updated: Jun 17, 2020 05:51AM UTC

Any updates Hannah?

Hardik | Last updated: Jun 17, 2020 06:08AM UTC

For more clarification of issue My sitemap(just for ex)looks as below: *http://mysite.com api v1 stats On scan with crawl and audit,urls that are getting scanned are with host "http://mysite.com" and url "/robots.txt", "/". Ideally it should also include url "/api/v1/stats"

Hardik | Last updated: Jun 17, 2020 09:19AM UTC

In items to scan list under Crawl and Audit option, there is only 1 url. i.e http://mysite.com In audit selected items all the urls are present. Can you comment.

Hannah, PortSwigger Agent | Last updated: Jun 17, 2020 01:02PM UTC

Hi Hardik Are you trying to scan an API? Burp is not able to scan an API automatically, and you will need to manually crawl followed by "Audit selected items" as you have been. You can find out more here: - https://portswigger.net/support/using-burp-to-enumerate-a-rest-api - https://portswigger.net/support/using-burp-to-test-a-rest-api

Hardik | Last updated: Jun 17, 2020 03:29PM UTC

So are we going to see the automatic scan of api in future release as in contents section all the items related to api is there.if we slect all the items there and send it to scan it will serve our purpose.

Hardik | Last updated: Jun 17, 2020 03:29PM UTC

So are we going to see the automatic scan of api in future release as in contents section all the items related to api is there.if we slect all the items there and send it to scan it will serve our purpose.

Hannah, PortSwigger Agent | Last updated: Jun 18, 2020 06:37AM UTC

Hi Hardik We do have plans in the future to allow users to automatically scan REST APIs. The current method to audit your API is to manually crawl it first to populate your sitemap, then "Audit selected items" to launch an audit. Have you had a look for any extensions that may help in the meantime? - OpenAPI Parser (https://portswigger.net/bappstore/6bf7574b632847faaaa4eb5e42f1757c)

Hardik | Last updated: Jun 18, 2020 10:07AM UTC

Can you suggest me way to automate this With burp rest-api, there are only 2 requests i.e post scan and get the scan using the location parameter. There is no option to post a scan with "crawl with selected items"

Hannah, PortSwigger Agent | Last updated: Jun 18, 2020 10:37AM UTC

Hi Hardik Yes, that is correct. From Burp's API you can only launch a "Crawl and audit".

Hardik | Last updated: Jun 21, 2020 01:26PM UTC

So, is there any way to achieve this automation stuff?

Hannah, PortSwigger Agent | Last updated: Jun 23, 2020 12:20PM UTC

Hi Hardik There is not currently any way to automate the scanning of API. We have plans in the future to adapt our scanner to enable it to automatically scan APIs. You may find the following extension helpful: https://portswigger.net/bappstore/6bf7574b632847faaaa4eb5e42f1757c

You must be an existing, logged-in customer to reply to a thread. Please email us for additional support.