Add support for .tar.zst to http_archive

In our project we’re using repository rules to download and uncompress prebuilt C++ libraries. Due to the large archive size we were evaluating different compression algorithms. One interesting candidate (which is also supported by CMake) is Zstandard. Right now it is not available within Bazel.

The Apache Commons Compress now supports zstd through the zstd-jni library. Unfortunately it relies on a native library.

I prepared a proof-of-concept branch with some shortcuts in gjasny/zstd-decompression and would like to ask if you’d consider adding support for Zstandard compression given the icky JNI implementation.

(With aircompressor there is native implementation available bit it lacks Big Endian and Stream support)

[brentleyjones]

@bazel-io fork 5.1

[1e100]

Here’s what I did for this: https://github.com/1e100/cloud_archive

This does require zstd support for tar, however, and does not rely entirely on built in facilities, but it can do authenticated downloads using the various cloud CLIs, validation, patching, and it does support Zstandard. And you don’t need to wait for 2 years to submit a patch. 😃