Research Academy
My account Customers About Blog Careers Legal Contact Resellers

Burp Suite User Forum

Create new post

crawl/URL's to Scan error

Jonathan | Last updated: Jul 30, 2019 04:14PM UTC

I'm having an issue (or is it a bug) whereby I have a website on a domain that has an underscore (for example, http://site_test.blah.com), but setting up a crawl scan type gives me 'Invalid URL to scan' error despite the site working through the proxy. Anyone else run into this?

Liam, PortSwigger Agent | Last updated: Jul 31, 2019 08:01AM UTC

The _ character cannot be used in a domain name.

Burp User | Last updated: Aug 01, 2019 01:29PM UTC

The _ character can be used if it's a cname

Rose, PortSwigger Agent | Last updated: Aug 05, 2019 12:32PM UTC

Jonathan, as this is not a valid domain name we don't support this. As a workaround, please add a record to your hosts file without an underscore to resolve to the site's IP address. Please let us know if you need any further assistance.

Burp User | Last updated: Aug 30, 2019 03:50PM UTC

Hi Rose - this is false. Please escalate this issue to your developers Please see this link for a more in depth explanation. The workaround wont work for Https either. https://stackoverflow.com/questions/2180465/can-domain-name-subdomains-have-an-underscore-in-it

Mike, PortSwigger Agent | Last updated: Sep 02, 2019 07:20AM UTC

Hi Sherwin, After investigation with our development team, it appears that our validation for the provided URL's are failing due to that character being present. We acknowledge that browsers do support this syntax within URL's and we want to match browser flexibility so I have logged a request with our development team to amend this validation. We will notify this thread once this change is available, unfortunately we can't provide an ETA.

Burp User | Last updated: Sep 03, 2019 05:34AM UTC

Hello. Just wanted to add that we are having the same issue as well and would very much appreciate a fix for this in a future release. Thanks,

You must be an existing, logged-in customer to reply to a thread. Please email us for additional support.