Jib should avoid parallel image downloads for same image
See original GitHub issueAlthough jib is thread-safe, it should be smarter.
Jib doesn’t currently lock the base image cache when downloading a base image, but instead downloads into a temporary directory and then attempts to moves the downloaded image into place. There is no locking to block other threads. So a Maven project with N modules that build images using the same base image (like gcr.io/distroless/java
) may result in N simultaneous pulls of the same image. Maybe we should provide a component to centralize downloading images?
_Originally posted by @briandealwis in https://github.com/GoogleContainerTools/jib/issues/1904#issuecomment-521352655_
Issue Analytics
- State:
- Created 4 years ago
- Comments:16 (10 by maintainers)
Top Results From Across the Web
Multi-module maven project with parallel builds fail with ... - GitHub
Environment: Jib version: 3.2.1 Build tool: Maven - 3.8.5 OS: Linux Ubuntu/MacOS ... Jib should avoid parallel image downloads for same image #2007....
Read more >GitLab CI: reuse the same image in two independent parallel ...
The first job will download the image from the docker hub and the next ones will get from the local docker service.
Read more >The basics of CI: How to run jobs sequentially, in parallel, or ...
It looks like there's a lot of public images around. So we can just grab one for our technology stack. It makes sense...
Read more >Parallels Image Tool
Virtual machines use virtual hard disks that actually are hard disk image files. ... installing and using it are the same as for...
Read more >Images - Kubernetes
Your container runtime can notice that the image layers already exist on the node so that they don't need to be downloaded again....
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
And this only matters when you enable the Maven parallel builds (
-T <thread counts>
) or run multiple Jib processes simultaneously on the same host. (And only when the base image layers have never been cached.) Does not affect correctness even in those cases.That is true, but when you don’t target a registry, Jib still caches base images locally to skip unnecessary pulls. So I think this issue is really only a problem for clean multi-module builds.