question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

GitLab CI: which directories can safely be cached?

See original GitHub issue

Some background Right now, our GitLab CI pipeline takes ~ 10 minutes to complete. The generation of thumbnails seems to be a major factor:

[..]
success Building production JavaScript and CSS bundles - 58.614s
success Rewriting compilation hashes - 0.001s
success run queries - 77.167s - 42/42 0.54/s
success Generating image thumbnails - 232.075s - 760/760 3.27/s
success Building static HTML for pages - 15.758s - 36/36 2.28/s
info Done building in 272.523996685 sec
[..]

Build times are reduced drastically when .cache and public are cached in .gitlab-ci.yml:

[..]
success Building production JavaScript and CSS bundles - 29.286s
success run queries - 29.868s - 2/2 0.07/s
success Building static HTML for pages - 7.725s - 36/36 4.66/s
info Done building in 56.332283976 sec
[..]

Question In all Gatsby-related .gitlab-ci.yml-examples I’ve seen so far, only node_modules/ is cached. This leaves me wondering if it’s safe to cache .cache and public in CI. If it’s safe to do so, why doesn’t this example list those folders?

Long story short: can both .cache and public safely be cached in a GitLab CI environment?

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
jonniebigodescommented, Dec 29, 2019

a while ago i encountered a issue with some stale images and one page that shouldn’t exist because the it’s source content was no longer present. Don’t know if that’s the case still i honestly haven’t had the time to test it thoroughly, probably someone else more knowledgeable can give some further insights on this.

0reactions
jooolacommented, Jan 5, 2021

@sidharthachatterjee Hello, something seems inconsistent on whether to cache the public and/or .cache folders. In my Gitlab CI build logs I see the following:

[...]
info We've detected that the Gatsby cache is incomplete (the .cache directory exists
but the public directory does not). As a precaution, we're deleting your site's
cache to ensure there's no stale data.
success initialize cache - 0.063s
success copy gatsby files - 0.026s
success onPreBootstrap - 0.010s
success  gatsby-source-wordpress  ensuring plugin requirements are met - 0.730s
⠀
info  gatsby-source-wordpress  
	This is either your first build or the cache was cleared.
	Please wait while your WordPress data is synced to your Gatsby cache.
	Maybe now's a good time to get up and stretch? :D
[...]

So either the previous PR #22501 is missing a folder, or the deletion of the .cache folder is not correct.

What should we do ?

EDIT: Nevermind I didn’t look at the full PR changes, a last commit did add the public folder.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Caching in GitLab CI/CD
All caches defined for a job are archived in a single cache.zip file. The runner configuration defines where the file is stored. By...
Read more >
GitLab CI: Cache and Artifacts explained by example
It's a set of files that a job can download before running and upload after execution. By default, the cache is stored in...
Read more >
Index · Caching · Ci · Help · GitLab - ETSI Forge
Where the caches are stored ; Shell, Locally, under the gitlab-runner user's home directory: /home/gitlab-runner/cache/<user>/<project>/<cache-key>/cache.zip .
Read more >
Cache dependencies in GitLab CI/CD
Where the caches are stored ; Shell, Locally, stored under the gitlab-runner user's home directory: /home/gitlab-runner/cache/<user>/<project>/<cache-key>/cache.
Read more >
Index · Caching · Ci · Help · GitLab
GitLab CI /CD provides a caching mechanism that can be used to save time when your jobs are running. Caching is about speeding...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found