Feature request: external dependency caching using remote gRPC protocol

Description of the feature request:

As part of implementing a proof of concept of a distributed cache for Bazel builds (see mwitkow/bazel-distcache) it seems that the remote_protocol.proto CASService is not used for caching <output_dir>/external content.

The reason why we’re interested in building distacache is because we run Dockerized bazel builds (using Jenkins and Concourse), and we’d rather not share the output_dir verbatim.

Not having the external cachable is a massive problem for such a use case. We have quite a few git dependencies for rules_go since there are tons of external dependencies in Go, and quite a few Skylark rules are referenced this way. Moreover, we have quite a few Maven deps which would be cacheable easily.

There seems to already be a RepositoryCache implementation in Bazel that allows HttpDownloader and MavenDownloader to cache from local disk (using --experimental_repository_cache) as part of https://github.com/bazelbuild/bazel/issues/1752. I think it could rely on the CASService, as it even has the appropriate hashing methods.

The proposal: (a): Extend RepositoryCache to be able to use the CASService, by reusing RemoteActionCache. (b): Come up with a way to use RepositoryCache for GitRepos. Most likely split them up to be a ZIP that can be cached like http_archive. Most likely belongs in a separate ticket.

CCs: @ola-rozenfeld - for RemoteCache @jin - for RepositoryCache @kchodorow - since seems to be main stakeholder in #1752 @dinowernli - since he probably wants to be on this ticket anyway 😉

[rockwotj]

Any updates here?

[rockwotj]

@pl-misuw see https://github.com/buchgr/bazel-remote and https://github.com/bazelbuild/bazel/pull/10622

In short you should be able to do this today.