Tests with "exclusive" tag do not get results from remote cache

Description of the problem / feature request / question:

I am trying out the remote cache functionality described here. Currently I’m running against a local hazelcast instance. This works great most of the time. One thing I noticed is that tests with the exclusive tag, despite being cached as normal in the local cache, will not seem to use the remote cache and will always rerun if there is no passing result in the local cache. This seems like an oversight?

If possible, provide a minimal example to reproduce the problem:

BUILD

cc_test(
  name = "test",
  srcs = ["test.cc"],
  tags = ["exclusive"], # comment this out to fix the problem
)

test.cc

#include <chrono>
#include <thread>

int main() {
  std::this_thread::sleep_for(std::chrono::seconds(10));
  return 0;
}

.bazelrc

startup --host_jvm_args=-Dbazel.DigestFunction=SHA1

build --spawn_strategy=remote
build --experimental_strict_action_env
build --remote_rest_cache=http://localhost:5701/hazelcast/rest/maps/cache

test --spawn_strategy=remote
test --experimental_strict_action_env
test --remote_rest_cache=http://localhost:5701/hazelcast/rest/maps/cache

Environment info

• Operating System: Ubuntu

• Bazel version (output of bazel info release): release 0.5.3

• Hazelcast version 3.8.6

Anything else, information or logs or outputs that would be helpful?

Here’s an example run. The second test will pass in 0.0s if either the exclusive tag is removed, or the second bazel clean command is omitted.

$ bazel clean && bazel test //... && bazel clean && bazel test //...                                         
INFO: Reading 'startup' options from /home/stu/tmp/.bazelrc: --host_jvm_args=-Dbazel.DigestFunction=SHA1
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.
INFO: Reading 'startup' options from /home/stu/tmp/.bazelrc: --host_jvm_args=-Dbazel.DigestFunction=SHA1
INFO: Analysed target //:test (9 packages loaded).
INFO: Found 1 test target...
Target //:test up-to-date:
  bazel-bin/test
INFO: Elapsed time: 10.405s, Critical Path: 10.20s
INFO: Build completed successfully, 7 total actions
//:test                                                                  PASSED in 10.0s

Executed 1 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
INFO: Reading 'startup' options from /home/stu/tmp/.bazelrc: --host_jvm_args=-Dbazel.DigestFunction=SHA1
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.
INFO: Reading 'startup' options from /home/stu/tmp/.bazelrc: --host_jvm_args=-Dbazel.DigestFunction=SHA1
INFO: Analysed target //:test (9 packages loaded).
INFO: Found 1 test target...
Target //:test up-to-date:
  bazel-bin/test
INFO: Elapsed time: 10.209s, Critical Path: 10.02s
INFO: Build completed successfully, 7 total actions
//:test                                                                  PASSED in 10.0s

Executed 1 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

[stepango]

Is there any timelines to get it fixed? If not how about introducing the flag to enable #8983 conditionally, in this case many teams could benefit from exclusive tests caching as well as keep internal tests green. Thoughts?

[JayThomason]

Here is a concrete use-case for why this bug is important: We have a lot of tests that require usage of the GPU. These tests use the exclusive tag to prevent OOM issues that can arise when running multiple tests that require the GPU at once. Because of this bug these tests have to run every single time that we do a test CI build.

In my opinion this bug breaks one of the core features of bazel: targets are being built which do not need to be built because the dependencies did not change.