HTTP request charset support.
I have 2 questions regarding charset support in the Content-Type header of a request. This relates to Burp community edition version v1.7.36
1. When charset is set to UTF-16 BE BOM, Burp seems to properly detect it only if the content-type is application/json. It fails to do so if it's text/html, text/plain. Any idea why?
2. Is there a list of all the charsets supported by Burp?
I should also add that this is working as expected if the charset is specifically set to ISO-8859-1 but not for UTF-16 when the content-type is text/plain. Based on this behavior, looks like the default charset Burp expects is UTF-8.
Unfortunately, Burp’s support for UTF-16 is very limited at present. This is something that we intend to work on in future, but this is going to be part of a major update to the message editor, rather than a quick fix. In principle it would be possible to code an extension to workaround this.
You can see a list of supported charsets in User options > Display > Character Sets. The “Use a specific character set” dropdown shows all the ones available. By default, Burp determines the character set from the Content-type header.