Increase single-thread scanner speed
Not sure if this is a bug or the standard behavior, so posting here first.
I tried this with burpsuite_pro_v1.6.11.jar and burpsuite_pro_v1.6.02.jar with the default initial config. The application was hosted locally with apache. Same results.
When I setup the scanner to do several items at the same time I see all the configured threads running simultaneously. Some of them go really fast, at multiple requests per second, but when there is only one thread active because it's the last one or because there is only one item to scan, the speed is reduced to approximately one per second.
Is there a way to speed up single-threaded scans for one item? I don't think it's related to the application, as I can see fast single-thread speeds under some circumstances, as described above.
Let me know if I can provide any further details.
We’re not able to reproduce any difference in timing of Burp Scanner’s requests based on the thread count, and we think the differences you observed are probably down to differences in the application’s response times for different requests. Since each scan item operates on the same base request, and is handled by a single thread, it is likely that different items will proceed at different speeds, because the application process some requests faster than others.
I've used the Logger++ extension to grab the request Burp Scanner was making at aprox. 1 request/s. I can then manually repeat it with cURL at faster speeds.
Is there any specific test you can think of to verify that the speed limitation is not caused by Burp? This might be an environment issue...
Can you try adding the header:
to the base request you are scanning, and see if that makes things go faster?
First I made the change in the proxy, adding a "Match and Replace" rule to set all requests with "Connection: close", and the browser requests increased in speed from 1/s to a few ms.
Then I scanned one of the slow requests sending it from Repeater with "Connection: close", and it was instant.
I then disabled KeepAlive in the apache configuration, and that made ALL the requests fly.
You nailed it!
I'm now just trying to figure out how I can make the Scanner set this for me when I start scans with the Scanning Wizard. Do you know if this is possible?
Thank you, you guys are awesome.
Glad you got things working. This issue comes up occasionally with some configurations of target servers. We plan to add some new options to Burp shortly, to set the Connection: close header. In the meantime, if you use the Proxy match/replace rule that you have, and also configure the Connection: close header in the Spider headers options, then the requests that are sent to the Scanner will already have the required header.
In today’s update to Burp, the Proxy by default sets the “Connection: close” header on all incoming requests, so all normal requests that reach the Scanner are now setting this header.
This question has received the maximum number of answers.