Research Academy
My account Customers About Blog Careers Legal Contact Resellers

Burp Suite User Forum

Create new post

Burp Enterprise and GitLab CI/CD Scan Error: Scan has failed: Could not connect to any start URLs.

Ron | Last updated: Apr 19, 2024 06:33PM UTC

Hello, I am setting up CI/CD DAST scanning in Gitlab using Burp Suite Enterprise and I'm having an issue I'm hoping someone can help with. The scanner kicks off but is not able to reach any endpoint and displays this error in the pipeline: - Scan started - Crawl phase started - Crawling: 0 locations visited - Scan has failed: Could not connect to any start URLs. - Scan failed to complete - refer to the log files for more information. The enterprise server uses a self signed certificate which is stored as a CI/CD variable, and the enterprise server is reachable by the scanner. The enterprise API key is stored as a CI/CD variable. The scanner successfully finds the "burp_config.yml" file which is very limited in scope for testing. All our Proxy variables are set in GitLab, and testing by adding a wget request in the enterprise scan container shows that wget has access to the same site we are trying to scan and is using the correct Proxy. All the examples I have looked at to configure Burp Enterprise and Gitlab show to use Docker in Docker which is not ideal in our scenario using Kubernetes. I have not been able to get the scanner to communicate with any host. - Is there some config I am missing? - Is the below script in the pipeline correct to kick off the scan or am I missing something there: "/usr/local/burpsuite_enterprise/bin/initiate-scan" - Where is the log file that is mentioned in the error above? - Does the scanner requires its own proxy settings outside of the settings configured for the Gitlab runner, and if so how are they set? The pipeline is very simple and looks like this: pipeline: image: name: private_registry.com/enterprise-scan-container:latest entrypoint: [""] stage: test allow_failure: true script: - "/usr/local/burpsuite_enterprise/bin/initiate-scan" artifacts: when: always paths: - burp_junit_report.xml reports: junit: burp_junit_report.xml The config file is also very simple and looks like this: enterpriseServer: url: https://enterprise.server.url reporting: reportFilePath: burp_junit_report.xml reportFormats: - JUNIT threshold: enabled: false minimumConfidence: TENTATIVE minimumSeverity: LOW scanConfigurations: builtIn: - Audit checks - passive site: startUrls: - https://urltoscan.com Any help is greatly appreciated. Thanks!

Thomas, PortSwigger Agent | Last updated: Apr 22, 2024 09:28AM UTC

Thank you for letting us know about your issue with CI-Driven scans. To look into this issue we need to see the Docker run command, pipeline output, and debug.log file for the scan. The debug.log file should be outputted in the workspace or working directory specified in your Docker run command. Additionally, does the web application being scanned have any authentication requirements such as SSO or NTLM? If so, has this been added to the scan? Please email support@portswigger.net with the requested information and attach the initiate-scan script, debug.log file, pipeline error output, and the output of the wget command specified.

Ron | Last updated: Apr 24, 2024 01:46PM UTC

Thanks for your reply. I have sent a support email and I'm waiting for a response. Regarding the above connectivity, I think this may be related to the upstream proxy config. If I understand the documentation correctly, the upstream proxy can be set in the burp_config.yml file. I am using the example provided by Portswiigger: https://github.com/PortSwigger/ci-cd-platform-scanning-examples/blob/main/ci-driven-scan-config-template.ym In this example there is a "proxies" section which can be used to set proxy details. burp_config.yml "proxies" example from above template: ----------------------- proxies: # Enter a list of proxy server configurations # - authentication: # type: Choose from None, Basic, NTLM v1, or NTLM v2. # username: username # password: password # domain: domain # domainHostname: hostname # destinationHost: localHost # proxyHost: host # proxyPort: 8080 ----------------------- I have tried to set this many times, and the pipeline job always fails due to a scan error related to the proxies configuration - "Unrecognised field in config file section: proxies" Job failure error: ----------------------- Executing "step_script" stage of the job script $ /usr/local/burpsuite_enterprise/bin/initiate-scan Found config at path /burp_enterprise/burp_config.yml You have not set a valid configuration. You can configure the container either by passing in environment variables, or by placing a burp_config.yml file in the container's working directory. See PortSwigger's online documentation for more information. Unrecognised field in config file section: proxies ----------------------- The proxy in use does not require authentication so I have commented that out of the config. My proxy config: ----------------------- proxies: # Enter a list of proxy server configurations #- authentication: # type: None # username: # password: # domain: domain # domainHostname: hostname destinationHost: www.site_to_scan.com proxyHost: http://proxy_address.com proxyPort: 80 ----------------------- I have tried to set this in many different ways, including setting authentication to see what happens, and I always get the "Unrecognised field in config file section: proxies" error unless all the keys are commented out except the initial "proxies" key. The debug.log file does not provide any further details. The documentation on setting upstream proxies is lacking. - Is this the correct way to set upstream proxies? - Is there something wrong with the syntax which causes this error? - Is there a bug which is causing this error? - Is there some way or an additional log to see what field is unrecognized? - Is there any further technical documentation that describes all of the config options? - Any other suggestions? I need to get CI/CD scans running ASAP so any help is appreciated. Thanks, Ron

Thomas, PortSwigger Agent | Last updated: Apr 25, 2024 12:27PM UTC

Hi All, We have looked into this with the information provided on this case, and the emailed in thread. At the date of this message, 25th April, the template yaml appears to have been setup incorrectly. The correct values for an authentication type of NONE on a proxy setup are as follows: proxies: # Enter a list of proxy server configurations - authentication: authType: NONE # username: username # password: password # domain: domain # domainHostname: hostname destinationHost: ginandjuice.shop proxyHost: 127.0.0.1 proxyPort: 80 To summarise, the "type" section was incorrect and needs to be "authType", while the value needs to be uppercase. Additionally, authType is a required field. We will be fixing this shortly after this message.

You must be an existing, logged-in customer to reply to a thread. Please email us for additional support.