Skip to content

Support brotli encoding in HAR capture #739

Open

Description

@mike10004

When capturing responses for a HAR, response content that is brotli-compressed (identified by a Content-Encoding header with value "br") is not decompressed. This results in the HAR containing base64-encoded content for those responses where it would be helpful to have plain text. Scripts and stylesheets are frequently served with brotli compression.

Currently, the ServerResponseCaptureFilter class only supports decompressing gzip-compressed response content in the decompressContents() method:

protected void decompressContents() {
    if (contentEncoding.equals(HttpHeaders.Values.GZIP)) {
        try {
            fullResponseContents = BrowserMobHttpUtil.decompressContents(getRawResponseContents());
            decompressionSuccessful = true;
        } catch (RuntimeException e) {
            log.warn("Failed to decompress response with encoding type " + contentEncoding + " when decoding request from " + originalRequest.getUri(), e);
        }
    } else {
        log.warn("Cannot decode unsupported content encoding type {}", contentEncoding);
    }
}

It would be helpful to support decompression of brotli-compressed content in this class, and moreover, it would be helpful to support creation of a custom ServerResponseCaptureFilter in the HarCaptureFilter constructor, perhaps by providing a factory as a constructor parameter.

There are some other technically valid content encodings that are not supported by the current server response filter; for example, the Content-Encoding header might be "identity" or might specify multiple encodings. The factory approach would allow Browsermob users to implement their own response content manipulations as needed.

The library org.brotli:dec:0.1.1 provides a pure-Java brotli decompression implementation that could be used for this purpose. I'd be glad to try out a patch to support brotli and submit a pull request for the functionality suggested here.

(Tested with version 2.1.5.)

Activity

  1. n0m0r3pa1n commented on Apr 18, 2018

    @n0m0r3pa1n

    Hi @mike10004 ,
    I saw this right ticket a minute ago. I saw the brotli problem because I wanted to get some Facebook data and I did this patch: #742 👍 You can try if it works for you, works fine on my side :)

  2. mike10004 commented on Apr 18, 2018

    @mike10004
    Author

    Thanks! That looks like a good implementation -- I'm just waiting to see what the browsermob-proxy developer says about the factory idea.

  3. jrgp commented on Jun 7, 2018

    @jrgp

    Hi folks, any traction/update on this? I think that PR looks good. 👍

  4. ericbeland commented on Jun 3, 2019

    @ericbeland

    It seems that development on the browsermob proxy has ceased. We have a fork, the browserup proxy that has merged brotli support. It should be a drop in replacement. https://github.com/browserup/browserup-proxy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions