Research Academy
My account Customers About Blog Careers Legal Contact Resellers

Burp Suite User Forum

Create new post

Not able to Crawl the application URLs which are behind the authentication and Not able to start embedded browser

Madhu | Last updated: Jun 09, 2020 01:09PM UTC

Team, When I run a scan from my enterprise with "Crawl strategy - most complete", it could able crawl for very few locations (in my case it's 4). In the event log I found a debug statement "Did not find a login form". My Application was written in JSP with standard form tag with action attribute & input submit button. Does crawling expects any special keywords/tags/names/ids in the page to identify the that it's login page? If I try running the scan with embedded browser, Scan failed with "Crawl was configured to use embedded browser, but a browser could not be started". Burpsuite & agent installed on CentOS – 7.7 Can you please help me in resolving the above issues. So that I can do further analysis. Thanks.

Hannah, PortSwigger Agent | Last updated: Jun 09, 2020 02:58PM UTC

You could try reinstalling Chrome on your system, as you may be missing some libraries. If this is the case, this should be evident in your enterpriseAgent logs. Could you try running the following commands on the Agent machine? 1. wgethttps://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb 2. sudo dpkg -i google-chrome*.deb 3. sudo apt-get -f install If this doesn't work, could you provide us with the following information: - The number of cores available on your Agent machine - The amount of RAM it has - The number of logical agents you have installed on there (this is the number of assigned agents that you can view by going to your Agents page in the Enterprise UI).

Madhu | Last updated: Jun 09, 2020 06:11PM UTC

As mine is CentOS, I've used the below commands to install the latest Chrome. 1. wget https://dl.google.com/linux/direct/google-chrome-stable_current_x86_64.rpm 2. sudo yum localinstall google-chrome-stable_current_x86_64.rpm After that i've restarted the agent service 1. systemctl restart burpsuiteenterpriseedition_agent.service Both agent and web-server are running on the same machine. Below are details about agent: 8-cores 16 GB RAM One Agent I've noticed an exception in the enterpriseAgent.log 2020-06-09 10:21:53 INFO net.portswigger.enterprise.agent.e.f - Received metric CRAWLING 0 0 [qtp1770324896-59] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - Exception report: [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - Category: UNEXPECTED [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - Detail: [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - burp.dyu: net.portswigger.devtools.client.a1: Unable to start browser: [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - [0609/102150.668185:FATAL:zygote_host_impl_linux.cc(116)] No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/master/docs/linux/suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox. [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - [0609/102151.715173:FATAL:zygote_host_impl_linux.cc(116)] No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/master/docs/linux/suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox. [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - [0609/102153.770682:FATAL:zygote_host_impl_linux.cc(116)] No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/master/docs/linux/suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox. [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.c__.a(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.d_6.<init>(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.ch7.a(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.f4d.e(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.f4d.<init>(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.fkk.a(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.fkk.a(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.ejv.a(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.eju.b(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.eju.a(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.eju.a(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.bt.lambda$startNextPhase$0(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.b2g.run(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.gen.run(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at java.base/java.lang.Thread.run(Thread.java:844) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - Caused by: net.portswigger.devtools.client.a1: Unable to start browser: [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - [0609/102150.668185:FATAL:zygote_host_impl_linux.cc(116)] No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/master/docs/linux/suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox. [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - [0609/102151.715173:FATAL:zygote_host_impl_linux.cc(116)] No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/master/docs/linux/suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox. [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - [0609/102153.770682:FATAL:zygote_host_impl_linux.cc(116)] No usable sandbox! Update your kernel or see https://chromium.googlesource.com/chromium/src/+/master/docs/linux/suid_sandbox_development.md for more information on developing with the SUID sandbox. If you want to live dangerously and need an immediate workaround, you can try using --no-sandbox. [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at net.portswigger.devtools.client.c.a(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at net.portswigger.devtools.client.c.a(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.gjg.b(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.gjg.a(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at burp.c__.a(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - ... 17 more [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - Caused by: java.io.IOException: Failed to read devtools port from browser output (bkunspv4da9zhz0fjo5a1j20wcv78w6uipxxqg9myz0yi0nf4ewtez498mmbpji03mbg6ptajyr7as1y8hd1xyyco9k4emy08rl9jggdfeoemds3tu) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at net.portswigger.devtools.client.u.<init>(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - at net.portswigger.devtools.client.av.a(Unknown Source) [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - ... 22 more [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - -------------------------------------------------------------------------------- [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - [c] 2020-06-09 10:21:53 INFO n.p.e.B.8.68aa15a1-767b-47ca-af2c-c3695d1bc530 - 2020-06-09 10:21:53: Audit failed! Crawl was configured to use embedded browser, but a browser could not be started [c] Thanks.

Hannah, PortSwigger Agent | Last updated: Jun 10, 2020 11:07AM UTC

It looks like you will need to either disable the sandbox, or change some environmental variables to allow you to run Chromium in a sandbox. We would not recommend disabling the sandbox, and do not currently have a way in the Enterprise UI to do this. This means that if you wish to disable the sandbox, the configuration will need to be exported from Burp Suite Professional. We recommend following this guide in order to be able to run the browser in a sandbox: https://superuser.com/questions/1294215/is-it-safe-to-enable-user-namespaces-in-centos-7-4-and-how-to-do-it In particular, you will need to run “echo 10000 > /proc/sys/user/max_user_namespaces” This should resolve your issue. We've created a ticket so that in future, we can better detect when this issue occurs and give appropriate instructions to fix it.

Madhu | Last updated: Jun 15, 2020 06:12PM UTC

Thanks Hannah, It resolved the issue.

Hannah, PortSwigger Agent | Last updated: Jun 16, 2020 07:52AM UTC

I'm glad to hear it! Please let us know if you need any further assistance.

You must be an existing, logged-in customer to reply to a thread. Please email us for additional support.