Research Academy
My account Customers About Blog Careers Legal Contact Resellers

Burp Suite User Forum

Create new post

Burp 2 - v2.1.06 - Scan / Crawl sends four times the same HTTP request for each entry

Bruno | Last updated: Nov 23, 2019 07:01AM UTC

Hello, While doing I scan / crawl of a website, I noticed that Burp 2 makes 4x time the same HTTP requests for each crawl action. for instance it will query /robots.txt four times, this happens also when setting the thread pool to use max one concurrent connection Sample screenshot is available here : https://imgur.com/a/WVgAegg

Liam, PortSwigger Agent | Last updated: Nov 25, 2019 09:38AM UTC

The crawler employs multiple crawler "agents" to parallelize its work. Each agent represents a distinct user of the application navigating around with their own browser. You can read more about how the crawler works on our website: - https://portswigger.net/burp/documentation/scanner/crawling

You must be an existing, logged-in customer to reply to a thread. Please email us for additional support.