question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

'Job A' unable to access cache of downstream 'Job B'

See original GitHub issue

Heya! I have these two jobs where Job B needs Job A: https://github.com/returntocorp/semgrep/blob/a9e8b90c2e3de705c54ca311b414608ffaa32ff7/.github/workflows/tests.yml#L15-L67

Both jobs do this:

        uses: actions/cache@v3
        with:
          path: ocaml-build-artifacts.tgz
          key: foo

But only Job B ever builds that artifact, while Job A only tries to restore it to see if Job B can be skipped altogether.

Now, I don’t see why this wouldn’t work, but it does not, see these logs:

I would expect the restore to succeed in Job A as well.

Issue Analytics

  • State:closed
  • Created a year ago
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
bishal-pdMSFTcommented, Aug 19, 2022

@underyx this is happening because of two different OS containers. Due to that, in Job A case zstd is used for compression. Can be inferred from logs

##[debug]Checking zstd --version
##[debug]*** zstd command line interface 64-bits v1.5.2, by Yann Collet ***

In Job B, zstd is not found and hence it falls back to gzip

##[debug]Checking zstd --version
##[debug]Unable to locate executable file: zstd. Please verify either the file path exists or the file can be found within a directory specified by the PATH environment variable. Also check the file mode to verify the file is executable.

Whenever compression technique is different, the caches are treated as unique even though cache-key is same. Otherwise the de-compression will fail.

The fix is either to use same OS or ensure that same tools (e.g. zstd) is installed on both.

0reactions
Phantsurecommented, Dec 23, 2022

Have updated the error message to show other versions on cache miss if debug is on: https://github.com/actions/toolkit/blob/main/packages/cache/src/internal/cacheHttpClient.ts#L132 Closing this issue

Read more comments on GitHub >

github_iconTop Results From Across the Web

github - Share cache between distinct jobs - Stack Overflow
GitHub Action says that cache is restored successfully in job2 , however, in job2 I can't find my_file in the directory where I...
Read more >
Downstream pipelines have issues with predefined variables
In pipeline 295993300, we see the job created in the downstream project. The MY_DIR variable ( ${CI_PROJECT_DIR}/cache ) is not expanded. The ...
Read more >
Failed auto triggers for downstream jobs · Issue #7518 - GitHub
I wanted to highlight that we use set_pipeline: self in many of these pipelines. We usually require any other job to wait until...
Read more >
Deep Diving into CircleCI Workspaces
This post is a follow-on of our overview of persisting data in Workflows. To learn all about how to best use workspaces, caching...
Read more >
Guide: SSD Particle Caching in CryoSPARC
When you run jobs that have the Cache particle images on SSD option turned ... When installing CryoSPARC, you can use the parameter...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found