2.1.04 scanner stalling on pretty much every test
Hello, I've been trying to use the newer burp but so far I'm having great trouble making it actually perform its job effectively due to scans rarely finishing and having to frequently be "unstuck".
The pattern at the moment is for a scan to be started with some settings setup up to try and prevent lock-ups, at the moment the "handling application errors during audit" is set to skip remaining checks if there's 1 failure, and skip remaining insertion points if 1 failure is seen. It's set to pause a task only if 1,000 consecutive audit items fail. This is solely to try to prevent these dreadful lock-ups.
In Project Options I've dropped all the timeouts down to about 5 seconds, again to try to prevent lock-ups.
What essentially happens is that after a while, the scanner stalls. I go into audit items and find that 20 or so (however many parallel requests are permitted) items are marked as "Scanning" but nothing is happening. To get things going again I either have to change scanning resource to one that handles more simultaneous requests, or I have to select the stalled audit items, cancel them, then audit again.
As a result of the constant scanner stalls, active scans rarely get past phase 1 on anything but the smallest apps. I succeeded on getting a login form through all scanning phases recently and regarded this as a bit of a victory. Just a simple login form, that's all it could do without breaking.
This fixes things temporarily but it then just stalls again. I've done about 4 tests using the 2.x branch of burp but will be ripping it out and going back to 1.x for the next test until I get the OK from some of my colleagues who are persisting with it at the moment, the rest have gone back to 1.x for the same reason -- stalling scanners.
Do you have performance feedback enabled (User options > Misc > Performance feedback)? If so, could you provide us with your diagnostics (Help > Diagnostics)? We can check for excpetions related to specific extensions.
Failing that, the best way to locate the extension causing the issue would be to enable them one at a time.
This won’t affect the content of the debug information, if you can provide us with your Debug ID we can use that to locate the diagnostic information your Burp Suite installation is sending back to us from your testing.
If you enable your extensions one at a time, and run a scan on each enabled extension, it will allow you to identify which extension is causing issues when scanning.
java.runtime.name OpenJDK Runtime Environment
java.specification.name Java Platform API Specification
java.specification.vendor Oracle Corporation
java.vendor Oracle Corporation
java.vm.compressedOopsMode Zero based
java.vm.info mixed mode
java.vm.name OpenJDK 64-Bit Server VM
java.vm.specification.name Java Virtual Machine Specification
java.vm.specification.vendor Oracle Corporation
java.vm.vendor Oracle Corporation
sun.java.command com.install4j.runtime.launcher.UnixLauncher launch ccf7dac9 0 0 burp.StartBurp
sun.management.compiler HotSpot 64-Bit Tiered Compilers
Burp Version 2.1.04
Burp Browser Version 0.144
Burp Browser binaries /home/user1/Library/Apps/BurpSuite/Current/BurpSuitePro/burpbrowser/0.144
Code source /home/user1/Library/Apps/BurpSuite/Current/BurpSuitePro/burpsuite_pro.jar
Debug ID pfhvwl9vahzty6jkmw9j:mps6
JAR type Installer
Wsdler Extension type: Java
Site Map Fetcher Extension type: Python
Custom Extension type: Java
Content Type Converter Extension type: Java
Custom Logger Extension type: Java
JSON Decoder Extension type: Python
.NET Beautifier Extension type: Java
WSDL Wizard Extension type: Python
CMS Scanner Extension type: Java
Cloud Storage Tester Extension type: Python
J2EEScan Extension type: Java
Additional Scanner Checks Extension type: Python
Active Scan++ Extension type: Python
Additional CSRF Checks Extension type: Python
AuthMatrix Extension type: Python
Autorize Extension type: Python
Backslash Powered Scanner Extension type: Java
Bypass WAF Extension type: Java
CO2 Extension type: Java
Error Message Checks Extension type: Java
Freddy, Deserialization Bug Finder Extension type: Java
HTML5 Auditor Extension type: Java
HTTPoxy Scanner Extension type: Java
Headers Analyzer Extension type: Python
JSON Beautifier Extension type: Java
Java Deserialization Scanner Extension type: Java
Logger++ Extension type: Java
Retire.js Extension type: Java
SSL Scanner Extension type: Python
Session Auth Extension type: Python
Software Version Reporter Extension type: Java
Software Vulnerability Scanner Extension type: Java
WordPress Scanner Extension type: Python
Total memory 641,728,512
Max memory 3,110,076,416
Free memory 167,549,096
Number of processors 3
Debug ID is: pfhvwl9vahzty6jkmw9j:mps6
When it stalled this last time, was ‘Submit anonymous feedback about Burp’s Performance’ enabled (User options > Misc > Performance feedback)? The last data I can find from your installation is dated 7th November.
Also, can you try upgrading to the latest version 2.1.05 and starting a new scan with no extensions enabled, just to confirm that we see no errors at that point? If we just disable the extensions after it has failed it won’t necessarily point us at the cause of the problem.
I'll detach the machine from the test network now and plug it into the net and do the upgrade, that might upload the performance data.
I'll try to do a scan tomorrow with no extensions, it's tricky as the stall isn't that predictable.
Ian, thanks for trying that for us. Unfortunately, we still haven’t seen any debug information.
Can I ask, what number do you have set for “Pause the task if * consecutive items fail”?
Have you tried using Burp’s Resource Pool settings to throttle your scanning?
I'm on a fast internal network with a large app to test, so throttling shouldn't be needed, in fact I've created a new resource pool with 25 simultaneous requests. I've tended to do this to get scanning going again after it's stalled -- when it stalls, creating a new resource pool with more simultaneous requests than the last one had was one of the ways I'd get it unstalled.
As for the value of "pause the task", I've varied that through trying to get to the bottom of this but I don't have a definitive list of what I set it to and what happened. I've never noticed any difference even when I set it to quite a high number (I think I had it up to 1000 at one point). I've also dropped timeouts down to 20 seconds (who waits 5 minutes for a DNS request after all?) as this is a fast internal app. I also altered the two "skip remaining checks" to various values to try and stop it from grinding to a halt.
I suppose the best thing to do right now is to try and figure out why the debug information isn't uploading. What do we need to do this.
Also is there any way to get some more rapid support? As jobs are quite short, these long drawn-out problem solving sessions aren't going to help. It is a paid product after all so it would be nice to get some kind of progress on this.
Thanks for the additional informtaion Ian.
We’re now seeing feedback from your instance of Burp. We can’t see anything relating to Burp Scanner. Could you confirm that your debug ID remains the same?
When the scan stalls, could you send a screenshot of the Audit Items tab to email@example.com?
Regarding support, we have just added two new members to the team and have two more joining in the coming months. I appreciate that this doesn’t help with your current issue, however, we do take technical product support seriously and are actively working to improve this function.
Debug ID is pfhvwl9vahzty6jkmw9j:hgjg
I'll email a screenshot to the support address quoting that debug ID.
We will look into the issue once we have received your email.
I’ve picked up the email thread. We’ll respond via email.