question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Stream Docker image tarball to disk directly instead of in memory

See original GitHub issue

I’m attempting to create a tarball of an existing Docker image via the https://docker-py.readthedocs.io/en/stable/images.html save example. When attempting to write the data to disk, I get MemoryError during the writing process.

The image is 1.516GB so I’m assuming we are running out of memory (this process is within a container).

Can the save be done directly to disk and avoid being pushed into memory?

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:11 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
shin-commented, Dec 13, 2016

We should provide a streaming option. You can try the following code excerpt in the meantime:

import docker

client = docker.from_env()
img = client.images.get('busybox:latest')
url = client.api._url('/images/{0}/get', img.id)
res = client.api._get(url, stream=True)
with open('/tmp/img.tar', 'w') as f:
  for chunk in res.iter_content(chunk_size=None):
    f.write(chunk)
0reactions
shin-commented, Feb 13, 2018

Just wanted to say - I read the messages, and it’s on my radar - should be a simple fix. Thanks everyone for the reports.

Read more comments on GitHub >

github_iconTop Results From Across the Web

docker save - Docker Documentation
docker save: Produces a tarred repository to the standard output stream. ... Save one or more images to a tar archive (streamed to...
Read more >
Pull container images faster with partial pulls | Enable Sysadmin
The new storage model attempts to use disk space better and reduce memory consumption. Pulls may be more efficient and therefore quicker, too....
Read more >
Moving Docker Containers and Images Around - Giant Swarm
Docker supports two different types of methods for saving container images to a single tarball: docker export - saves a container's running or...
Read more >
Error processing tar file(exit status 1): unexpected EOF
docker -compose passes the context to the engine as a tar file, therefore, the build command was packing a tar (the .dump file)...
Read more >
Create and upload a Docker image with a Dockerfile
Overview Dockerfiles are text files that store the commands you would execute on the command line inside a container to create a Docker...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found