question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Add brotli compression

See original GitHub issue

Describe the feature Add brotli compression to Javalin.

Additional context We should probably also rename the current configuration (.disableDynamicGzip()) to something along the lines of

dynamicCompression: true

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:34 (23 by maintainers)

github_iconTop GitHub Comments

1reaction
nixxcodecommented, Jul 6, 2019

@tipsy @pkkummermo After scrapping BrotliHaxe due to the horrendous memory issues, I went back to JBrotli, but this time I forked it and started playing around in the source, rather than just including it in Javalin and following usage examples. I made some interesting discoveries.

The original bug I had problems with, that was crashing the JVM? It’s only an issue when using the BrotliStreamCompressor object from JBrotli. I’ve found that by converting our resultStream to a byte array and feeding that to the regular BrotliCompressor (non-stream version) object in a single chunk, everything works perfectly fine!

I am pretty confident this has fully resolved the problem, since I found the cause of the JVM crash here: https://github.com/google/brotli/issues/346 It is caused by a bug in the old version of Brotli (0.5.0), where a faulty boolean flag makes the Brotli data writer believe that it has reached the end of the data stream after being called twice. Since the non-stream version of JBrotli’s object compresses the entire byte array in a single pass rather than using a stream, it gets around the problem entirely.

So, I now have a functioning Brotli compressor built into Javalin, here: https://github.com/nixxcode/javalin/

Performance wise, it matches up to what I read in this article: https://blogs.akamai.com/2016/02/understanding-brotlis-potential.html

I have been testing with level 4 compression, result size on the 10k SillyObject test is 2235 bytes as opposed to Gzip’s 7740, so roughly a 9.2% improvement when compared against the uncompressed length of 59679 bytes.

Response times and memory usage are identical to those of Gzip, so performance is no longer an issue.

There are still some considerations, however:

  1. JBrotli’s compression is not a pure Java implementation. It uses JNI bindings to execute the compression code via native libraries and is thus platform dependent. That means it may not work on every possible platform, though I don’t think this is a huge issue, since we can still offer Gzip as a fallback.
  2. While being functional and having excellent performance, JBrotli is still using a Brotli library version that is 3 years out of date. Since everything seems to work great, I don’t necessarily think this is a problem, but I still wanted to mention it.

If you want to have a play with it, the branch I linked to earlier should build and run without any errors when cloned. I have created a test class named TestServer.kt with a runnable main() function, which keeps the Javalin server running until manually stopped. I use this to play around and test via web browser requests. Additionally, all the original/updated tests within TestCompression.kt are passing.

If you are happy to move forward with this, I will open a pull request. Alternatively, please let me know what else needs to be done. 😃

Thanks!

1reaction
tipsycommented, Jul 4, 2019

Honestly, I have no idea if I am in over my head, but there’s only one way to find out! I will give it a go.

Great! I’m not sure I can help much with the implementation, but feel free to use this issue for moral support 😄

Read more comments on GitHub >

github_iconTop Results From Across the Web

Brotli Compression: What It is and How to Use It in ...
Adding Brotli Compression Manually · Run this command: sudo apt update && sudo apt install brotli · Now you need to install the...
Read more >
Brotli Compression: A Fast Alternative to GZIP Compression
GZIP compression is one way to do this, but Brotli compression is an alternative fledgling method that commands attention.
Read more >
How To Enable Brotli Compression in Apache
Brotli compression is supported by all the major browsers like Chrome, Firefox, ... First, install the brotli package on your system.
Read more >
Minify and compress network payloads with brotli - web.dev
In this codelab, learn how Brotli compression can further reduce compression ratios and your app's overall size.
Read more >
Brotli compression using a reduced dictionary
Brotli is a state of the art lossless compression format, supported by all major browsers. It is capable of achieving considerably better ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found