question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Add support for `zstd` compression

See original GitHub issue

It’d be really nice to add support for zstd compression now that this is an officially supported codec (KIP-110).

Issue Analytics

  • State:open
  • Created 4 years ago
  • Reactions:8
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

3reactions
jeffwidmancommented, Apr 23, 2019

Would be really nice to refactor the codec compression interfaces into something more generic, allowing users to wrap their favorite library – rather than us having to pick a favorite to integrate with directly.

Agreed. However, there is still protocol-level considerations since the producer has to signal to the broker what compression format was used (which it, in turn, passes to downstream consumers).

What would be a good interface?

I’m thinking the consumer/producer can pass in a custom class. And that class has four pieces:

  1. compress()
  2. decompress()
  3. Class-level var indicating which protocol flag to set… allowed values are None/gzip/snappy/other broker-supported values
  4. Any custom configs that would be passed to the underlying library… such as the level of compression, multi-core vs single (this may not be relevant for our use case), etc. This seems to me to be the trickiest piece, how to do this somewhat generically/thin so it provides flexibility rather than straight-jacketing the user.
0reactions
einvincecommented, Jun 7, 2022
Read more comments on GitHub >

github_iconTop Results From Across the Web

[GIT PULL][PATCH v6 0/8] Add support for ZSTD-compressed ...
Hi all, This patch set adds support for a ZSTD-compressed kernel, ramdisk, and initramfs in the kernel boot process. ZSTD-compressed ramdisk and initramfs ......
Read more >
Add support for ZSTD-compressed kernel and initramfs
Add support for ZSTD -compressed kernel and initramfs ; Date: Wed, 8 Jul 2020 11:50:17 -0700 ; Message-ID: <20200708185024.2767937-1-nickrterrell@gmail.com>.
Read more >
Add support for Zstandard to System.IO.Compression #59591
The library supports regular compression levels from 1 up to ZSTD_maxCLevel(), which is currently 22. Levels >= 20, labeled --ultra , should be ......
Read more >
0003-Add-support-for-ZSTD-compressed-kernel.patch - aur.git
+config KERNEL_ZSTD + bool "ZSTD" + depends on HAVE_KERNEL_ZSTD + help + ZSTD is a compression algorithm targeting intermediate compression + with fast ......
Read more >
WL#12039: Add support for zstd compression to classic ...
The goal of this worklog is to add support to specify zstd compression algorithm in the network communication between client and server.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found