Does Not Work in Docker 1.10
See original GitHub issueIf you have Docker 1.10 installed, the format of the layers list command changes. Internally, the squash command relies on this output and the new output is not fully compatible with squash.
With Docker 1.9:
hayden@tayter-tot:~/gits/weber$ sudo docker --version
Docker version 1.9.1, build a34a1d5
hayden@tayter-tot:~/gits/weber$ sudo docker-scripts layers croscon/proj:dev
5549154b6b000b933097c461f5d367ef57f9fc5d3ccdddb77929699ce8eec86a
ββ dbcf09c742a7231a963b6e8765c3fe4cd917df10e981e7f224fb4df475258afe
ββ 40964da1bf475b05d38b509bc53e2304fd405af0fbfb7144157bf9aa18ebb4c8
ββ c29e52d44f6950f3cd33faf8d7745a552896b2a64c4c98110f92c23e6c1e0c8b
ββ 8f443e3bcacc8a75b8f0ac3a45680ed01884a6e567350341eff2cebf452f6063
ββ 5cb686eda5ece983a923051a5d189bfec509346b0b6c3a298aa6744f7ba31af3
ββ 4f1b6148c294845e4b3fef36539bafcf5263208206fcd7017eb8ec930baba6b7
ββ b4584fba539246f2530d154d7f3fe6650aec2cda2e5bd9eac4e1187112f48cd9
ββ ffc831064984dd4a91d420334958b4baa633281b520179029b0285f0526d20ce
ββ 9a1267d17938e7d3703a33b23fc95c4ff2f49efae58ca2a6bda10e0100fcdd17
ββ ac66319dcc481042e57a591810b8ffa41b1e822730d6e92b903b421b035e0ed6
ββ 006b881a418e2e138e5eb39ea77115b54e1f6b8ccd0be0bafc4710cd9743346c
ββ a0c02f0840575a63eab191e6cb7dfd8a8bcb7c6c1cb4bd6bfba20c628ed63379
ββ 7c91f289c345540c9b05f84d1201eab71892f26eb815ba5d69cddd7bddb83e58
ββ b0d46421d9cb9b26bda49d3f746a9a2452e41d03414218d085d5c3aa86ccc425
ββ c3c4a87b2a3b621cd72be81880cf9b17205aa060c4aa27d265e9bb1f54d34e0a
ββ 17cd26d4cd9c353e101587841f725b614590182e674ea8a4cf6d3aa98eaf87a3
ββ 3dd9146587db50ca4dde148a4d4cd581b6aef78d03b81dfb70d0562a60043cb8
ββ f82c8f9580e049ebbf1a4bfcf854c41fa60cd713f5ece881cdd2cfda3ad07eff
With Docker 1.10:
hayden@tayter-tot:~/gits/weber$ docker --version
Docker version 1.10.3, build 20f81dd
hayden@tayter-tot:~/gits/weber$ sudo docker-scripts layers croscon/proj:dev
sha256:55108c8348c54dd69c6269a18e8d3eee8196edc6636fcfce5e7fa360457df0d5
ββ sha256:96cccc5336f33cdd959d6bcabbcea74f614b8088983978082dffe4dcd4f0e143
ββ sha256:76ceed73cddf5ebc24d7f06d4e21fa07f4e7668500f692b120be7b5a70c59d0a
ββ sha256:32e0f98465b60e1828541079956be09f0f9208aad4a261140965e483a0d8be8f
ββ sha256:d853f9a04aba5dacd9364b6aa7bac8303291364f93d09e1f3c4c7a03bdd6df5e
ββ sha256:e9a498049959e2c9c34f61d9df61ba1efaa3dbc6ffd9ca7fd018761c92ed3981
ββ sha256:e753243f8129f1602810cfeef20004d4e4a3ef5ac80a30b920dd5057910e291d
ββ sha256:279c403da412190b54526baa60090dcac6bb61d75131bfdc289e13bb00b84781
ββ sha256:f2bcb93da3ed7cca47ce89c301358d0b4fa8437f300a0e0ddc791cae6bf2fba8
ββ sha256:b0f55883cc3f7a2c136b27b012c910ef86e1221d9928e61f3d7e7ff1cf45f72f
ββ sha256:9e185fa0f2f2ee1915b2ffd63110fd992a77ee2b19602ce7dc68cf40be326bad
ββ sha256:17ec8b8cb80ce3a7cc94ca4dce38298b36b23b487bc81476ce06484018d3bb29
ββ sha256:400184eceac65b9454d4f7c69fe8ac79dd44214f0da1b0cf824cc25d928da10e
ββ sha256:8f5a1053463993646984e18cdd803c241042c43d5a26ca7eec79d2e1b132ce80
ββ sha256:0f84bbbcbb50fc83ca25dc60d1dd6fe65df1ca2a6b68f86dd893863bce1e272d
ββ sha256:b730ce98d7421ab8b0128e2a428f193aee5037a6a53e261c96f8506b838fdb8b
ββ sha256:e04cd6ad8c2f73e127dc4657a56325842ef98fe5d25a039bf9120f918ba1425a
ββ sha256:fc7e129334445ba50c499fbb8319ad8640755e443648e5240c403a47e0d8841d
ββ sha256:25e73edb532b6ddbfc7956008bdcc22a25e6d41b82c5f9973c6ac6ff49ba2d19
Notice the new prefix of sha256:
And then when I run docker-scripts squash on Docker 1.10, we get:
hayden@tayter-tot:~/gits/weber$ sudo docker-scripts squash -f sha256:55108c8348c54dd69c6269a18e8d3eee8196edc6636fcfce5e7fa360457df0d5 -t dev_squashed croscon/proj:dev
2016-03-31 18:29:47,991 root INFO Running version 0.4.4, Docker 20f81dd, API 1.22...
2016-03-31 18:29:47,991 root INFO Squashing image 'croscon/proj:dev'...
2016-03-31 18:29:48,051 root INFO Old image has 19 layers
2016-03-31 18:29:48,051 root INFO Attempting to squash from layer sha256:55108c8348c54dd69c6269a18e8d3eee8196edc6636fcfce5e7fa360457df0d5...
2016-03-31 18:29:48,052 root INFO Checking if squashing is necessary...
2016-03-31 18:29:48,052 root INFO We have 18 layers to squash
2016-03-31 18:29:48,052 root INFO Saving image sha256:25e73edb532b6ddbfc7956008bdcc22a25e6d41b82c5f9973c6ac6ff49ba2d19 to /tmp/docker-squash-bDg3Vt/image.tar file...
2016-03-31 18:30:08,048 root INFO Image saved!
2016-03-31 18:30:08,051 root INFO Unpacking /tmp/docker-squash-bDg3Vt/image.tar tar file to /tmp/docker-squash-bDg3Vt/old directory
2016-03-31 18:30:09,454 root INFO Archive unpacked!
2016-03-31 18:30:09,455 root INFO New layer ID for squashed content will be: 42a8ac223df71242e2057b20ad1c0240b0f9223eb942889b12faee09d0189818
2016-03-31 18:30:09,455 root INFO Starting squashing...
2016-03-31 18:30:09,455 root INFO Squashing layer 'sha256:25e73edb532b6ddbfc7956008bdcc22a25e6d41b82c5f9973c6ac6ff49ba2d19'...
Traceback (most recent call last):
File "/usr/local/bin/docker-scripts", line 9, in <module>
load_entry_point('docker-scripts==0.4.4', 'console_scripts', 'docker-scripts')()
File "/usr/local/lib/python2.7/dist-packages/docker_scripts/cli.py", line 96, in run
cli.run()
File "/usr/local/lib/python2.7/dist-packages/docker_scripts/cli.py", line 88, in run
args.func(args)
File "/usr/local/lib/python2.7/dist-packages/docker_scripts/cli.py", line 31, in run_squash
from_layer=args.from_layer, tag=args.tag, output_path=args.output_path, tmp_dir=args.tmp_dir).run()
File "/usr/local/lib/python2.7/dist-packages/docker_scripts/squash.py", line 497, in run
layers_to_squash, layers_to_move, squashed_tar, old_image_dir)
File "/usr/local/lib/python2.7/dist-packages/docker_scripts/squash.py", line 337, in _squash_layers
with tarfile.open(layer_tar_file, 'r', format=tarfile.PAX_FORMAT) as layer_tar:
File "/usr/lib/python2.7/tarfile.py", line 1660, in open
return func(name, "r", fileobj, **kwargs)
File "/usr/lib/python2.7/tarfile.py", line 1722, in gzopen
fileobj = bltn_open(name, mode + "b")
IOError: [Errno 2] No such file or directory: u'/tmp/docker-squash-bDg3Vt/old/sha256:25e73edb532b6ddbfc7956008bdcc22a25e6d41b82c5f9973c6ac6ff49ba2d19/layer.tar'
Removing the sha256: from the -f layer does not help:
hayden@tayter-tot:~/gits/weber$ sudo docker-scripts squash -f 55108c8348c54dd69c6269a18e8d3eee8196edc6636fcfce5e7fa360457df0d5 -t dev_squashed croscon/proj:dev
2016-03-31 18:30:23,899 root INFO Running version 0.4.4, Docker 20f81dd, API 1.22...
2016-03-31 18:30:23,900 root INFO Squashing image 'croscon/proj:dev'...
2016-03-31 18:30:23,962 root INFO Old image has 19 layers
2016-03-31 18:30:23,962 root INFO Attempting to squash from layer sha256:55108c8348c54dd69c6269a18e8d3eee8196edc6636fcfce5e7fa360457df0d5...
2016-03-31 18:30:23,962 root INFO Checking if squashing is necessary...
2016-03-31 18:30:23,962 root INFO We have 18 layers to squash
2016-03-31 18:30:23,962 root INFO Saving image sha256:25e73edb532b6ddbfc7956008bdcc22a25e6d41b82c5f9973c6ac6ff49ba2d19 to /tmp/docker-squash-l476cz/image.tar file...
2016-03-31 18:30:44,348 root INFO Image saved!
2016-03-31 18:30:44,352 root INFO Unpacking /tmp/docker-squash-l476cz/image.tar tar file to /tmp/docker-squash-l476cz/old directory
2016-03-31 18:30:45,829 root INFO Archive unpacked!
2016-03-31 18:30:45,829 root INFO New layer ID for squashed content will be: 3af05c8f8a8158bcd12991c84097bb8c47635459745306046f861a126d23c0e1
2016-03-31 18:30:45,829 root INFO Starting squashing...
2016-03-31 18:30:45,829 root INFO Squashing layer 'sha256:25e73edb532b6ddbfc7956008bdcc22a25e6d41b82c5f9973c6ac6ff49ba2d19'...
Traceback (most recent call last):
File "/usr/local/bin/docker-scripts", line 9, in <module>
load_entry_point('docker-scripts==0.4.4', 'console_scripts', 'docker-scripts')()
File "/usr/local/lib/python2.7/dist-packages/docker_scripts/cli.py", line 96, in run
cli.run()
File "/usr/local/lib/python2.7/dist-packages/docker_scripts/cli.py", line 88, in run
args.func(args)
File "/usr/local/lib/python2.7/dist-packages/docker_scripts/cli.py", line 31, in run_squash
from_layer=args.from_layer, tag=args.tag, output_path=args.output_path, tmp_dir=args.tmp_dir).run()
File "/usr/local/lib/python2.7/dist-packages/docker_scripts/squash.py", line 497, in run
layers_to_squash, layers_to_move, squashed_tar, old_image_dir)
File "/usr/local/lib/python2.7/dist-packages/docker_scripts/squash.py", line 337, in _squash_layers
with tarfile.open(layer_tar_file, 'r', format=tarfile.PAX_FORMAT) as layer_tar:
File "/usr/lib/python2.7/tarfile.py", line 1660, in open
return func(name, "r", fileobj, **kwargs)
File "/usr/lib/python2.7/tarfile.py", line 1722, in gzopen
fileobj = bltn_open(name, mode + "b")
IOError: [Errno 2] No such file or directory: u'/tmp/docker-squash-l476cz/old/sha256:25e73edb532b6ddbfc7956008bdcc22a25e6d41b82c5f9973c6ac6ff49ba2d19/layer.tar'
So we poke around the filesystem in that tmp directory and see:
hayden@tayter-tot:~/gits/weber$ sudo ls -l /tmp/docker-squash-bDg3Vt/old
total 88
drwxr-xr-x 2 root root 4096 Feb 25 14:19 00480875bc1a4e5b7c61c748f73150d70e79763ee6519cf59c105d4412fa6604
drwxr-xr-x 2 root root 4096 Feb 25 14:19 1a64780f8db579faa981e531960dd6d2fe071cd20c6861b4a0b0fe2a6db33473
drwxr-xr-x 2 root root 4096 Feb 25 14:19 1e8634ab1ea8f04c0d8c0c66815e1dfc2e303114ac2cd794787b7b1a9a53fbfe
drwxr-xr-x 2 root root 4096 Feb 25 14:19 1f0dcbf0cd763568054353fa61ee009a27f17752e92785bc118d9475e3b4aa1c
drwxr-xr-x 2 root root 4096 Feb 25 14:19 2239fadacb56836575af9ecf4cf9077a5d3ffff3d7d00585165c276678a18b08
-rw-r--r-- 1 root root 8077 Feb 25 14:19 25e73edb532b6ddbfc7956008bdcc22a25e6d41b82c5f9973c6ac6ff49ba2d19.json
drwxr-xr-x 2 root root 4096 Feb 25 14:19 2d3b81eee53fae62cb5a04c8ac9192ba383b478f645607577c01f6d6b0cb7276
drwxr-xr-x 2 root root 4096 Feb 25 14:19 4af438cf458c3c6e5eefe90a3238c9dc312d4d95827b4702c49c2d6ef123934f
drwxr-xr-x 2 root root 4096 Feb 25 14:19 56367582146f55dcf8825f58e59395386a6ff9bc1354a880e90afbdb701fa2b4
drwxr-xr-x 2 root root 4096 Feb 25 14:19 57dda9f8be7378093e29d665dbfd8d98acd18c5d3e5a31f8389e5cc23aa0da17
drwxr-xr-x 2 root root 4096 Feb 25 14:19 5e250a6b5b8cd21cb9e16c6396206e526130c029be71e5784fd76d2074f0723b
drwxr-xr-x 2 root root 4096 Feb 25 14:19 63e9eb8285fe5e7212932f3eb78d85bfc41685cf1d7aefdf5afbd6b0e73bea6d
drwxr-xr-x 2 root root 4096 Feb 25 14:19 6b1c8a75b2bebed4e65ffb9d21669762e846160329f9300ff152de2b1da3495e
drwxr-xr-x 2 root root 4096 Feb 25 14:19 6f175e079c650f95487c21ad70d03c085d34b5bfb3adde729ccbbe6a9c4e03ef
drwxr-xr-x 2 root root 4096 Feb 25 14:19 81a2506886e93dd6808e0fc9d56ad720b916f743f31fc68b6fb53ffb1a4fd44b
drwxr-xr-x 2 root root 4096 Feb 25 14:19 ad8da3f6f98afd292a05e49dc9f63253907e1b9a0066c3ee049447d818d25b8f
drwxr-xr-x 2 root root 4096 Feb 25 14:19 c8ada7fa2e810589ab7a0749c3391e14c6b45586f338a34b9a2fa140d88b9f49
drwxr-xr-x 2 root root 4096 Feb 25 14:19 d60067a7ffe2239504f0ce672352600860500f5d83cfe59af1a49e11fb689617
drwxr-xr-x 2 root root 4096 Feb 25 14:19 eefe7f608aa7f35ac7a69a65923999b4b015bf5656f0d1400bb0c8c5054121b8
drwxr-xr-x 2 root root 4096 Feb 25 14:19 f14343db6862e571cd79afdb2726b9a4bc98e28a447ffae34c09633f7187af86
-rw-r--r-- 1 root root 1575 Dec 31 1969 manifest.json
And none of the layers are prefixed with sha256:.
Now with Docker 1.9:
hayden@tayter-tot:~/gits/weber$ sudo docker-scripts layers croscon/proj:dev
5549154b6b000b933097c461f5d367ef57f9fc5d3ccdddb77929699ce8eec86a
ββ dbcf09c742a7231a963b6e8765c3fe4cd917df10e981e7f224fb4df475258afe
ββ 40964da1bf475b05d38b509bc53e2304fd405af0fbfb7144157bf9aa18ebb4c8
ββ c29e52d44f6950f3cd33faf8d7745a552896b2a64c4c98110f92c23e6c1e0c8b
ββ 8f443e3bcacc8a75b8f0ac3a45680ed01884a6e567350341eff2cebf452f6063
ββ 5cb686eda5ece983a923051a5d189bfec509346b0b6c3a298aa6744f7ba31af3
ββ 4f1b6148c294845e4b3fef36539bafcf5263208206fcd7017eb8ec930baba6b7
ββ b4584fba539246f2530d154d7f3fe6650aec2cda2e5bd9eac4e1187112f48cd9
ββ ffc831064984dd4a91d420334958b4baa633281b520179029b0285f0526d20ce
ββ 9a1267d17938e7d3703a33b23fc95c4ff2f49efae58ca2a6bda10e0100fcdd17
ββ ac66319dcc481042e57a591810b8ffa41b1e822730d6e92b903b421b035e0ed6
ββ 006b881a418e2e138e5eb39ea77115b54e1f6b8ccd0be0bafc4710cd9743346c
ββ a0c02f0840575a63eab191e6cb7dfd8a8bcb7c6c1cb4bd6bfba20c628ed63379
ββ 7c91f289c345540c9b05f84d1201eab71892f26eb815ba5d69cddd7bddb83e58
ββ b0d46421d9cb9b26bda49d3f746a9a2452e41d03414218d085d5c3aa86ccc425
ββ c3c4a87b2a3b621cd72be81880cf9b17205aa060c4aa27d265e9bb1f54d34e0a
ββ 17cd26d4cd9c353e101587841f725b614590182e674ea8a4cf6d3aa98eaf87a3
ββ 3dd9146587db50ca4dde148a4d4cd581b6aef78d03b81dfb70d0562a60043cb8
ββ f82c8f9580e049ebbf1a4bfcf854c41fa60cd713f5ece881cdd2cfda3ad07eff
hayden@tayter-tot:~/gits/weber$ sudo docker-scripts squash -f 5549154b6b000b933097c461f5d367ef57f9fc5d3ccdddb77929699ce8eec86a -t dev_squashed croscon/proj:dev
2016-03-31 18:32:45,341 root INFO Running version 0.4.4, Docker a34a1d5, API 1.21...
2016-03-31 18:32:45,341 root INFO Squashing image 'croscon/proj:dev'...
2016-03-31 18:32:45,420 root INFO Old image has 19 layers
2016-03-31 18:32:45,420 root INFO Attempting to squash from layer 5549154b6b000b933097c461f5d367ef57f9fc5d3ccdddb77929699ce8eec86a...
2016-03-31 18:32:45,420 root INFO Checking if squashing is necessary...
2016-03-31 18:32:45,420 root INFO We have 18 layers to squash
2016-03-31 18:32:45,421 root INFO Saving image f82c8f9580e049ebbf1a4bfcf854c41fa60cd713f5ece881cdd2cfda3ad07eff to /tmp/docker-squash-LJTVAf/image.tar file...
2016-03-31 18:32:50,275 root INFO Image saved!
2016-03-31 18:32:50,278 root INFO Unpacking /tmp/docker-squash-LJTVAf/image.tar tar file to /tmp/docker-squash-LJTVAf/old directory
2016-03-31 18:32:52,107 root INFO Archive unpacked!
2016-03-31 18:32:52,108 root INFO New layer ID for squashed content will be: 951570c31961ac4f1bf8fa8ef0514145418d68b1fc48fc64014a6d07af737d86
2016-03-31 18:32:52,108 root INFO Starting squashing...
2016-03-31 18:32:52,108 root INFO Squashing layer 'f82c8f9580e049ebbf1a4bfcf854c41fa60cd713f5ece881cdd2cfda3ad07eff'...
2016-03-31 18:32:52,109 root INFO Squashing layer '3dd9146587db50ca4dde148a4d4cd581b6aef78d03b81dfb70d0562a60043cb8'...
2016-03-31 18:32:52,110 root INFO Squashing layer '17cd26d4cd9c353e101587841f725b614590182e674ea8a4cf6d3aa98eaf87a3'...
2016-03-31 18:32:52,110 root INFO Squashing layer 'c3c4a87b2a3b621cd72be81880cf9b17205aa060c4aa27d265e9bb1f54d34e0a'...
2016-03-31 18:32:52,345 root INFO Squashing layer 'b0d46421d9cb9b26bda49d3f746a9a2452e41d03414218d085d5c3aa86ccc425'...
2016-03-31 18:32:55,823 root INFO Squashing layer '7c91f289c345540c9b05f84d1201eab71892f26eb815ba5d69cddd7bddb83e58'...
2016-03-31 18:32:55,829 root INFO Squashing layer 'a0c02f0840575a63eab191e6cb7dfd8a8bcb7c6c1cb4bd6bfba20c628ed63379'...
2016-03-31 18:32:56,554 root INFO Squashing layer '006b881a418e2e138e5eb39ea77115b54e1f6b8ccd0be0bafc4710cd9743346c'...
2016-03-31 18:33:02,070 root INFO Squashing layer 'ac66319dcc481042e57a591810b8ffa41b1e822730d6e92b903b421b035e0ed6'...
2016-03-31 18:33:02,094 root INFO Squashing layer '9a1267d17938e7d3703a33b23fc95c4ff2f49efae58ca2a6bda10e0100fcdd17'...
2016-03-31 18:33:02,099 root INFO Squashing layer 'ffc831064984dd4a91d420334958b4baa633281b520179029b0285f0526d20ce'...
2016-03-31 18:33:14,909 root INFO Squashing layer 'b4584fba539246f2530d154d7f3fe6650aec2cda2e5bd9eac4e1187112f48cd9'...
2016-03-31 18:33:21,534 root INFO Squashing layer '4f1b6148c294845e4b3fef36539bafcf5263208206fcd7017eb8ec930baba6b7'...
2016-03-31 18:33:21,556 root INFO Squashing layer '5cb686eda5ece983a923051a5d189bfec509346b0b6c3a298aa6744f7ba31af3'...
2016-03-31 18:33:21,573 root INFO Squashing layer '8f443e3bcacc8a75b8f0ac3a45680ed01884a6e567350341eff2cebf452f6063'...
2016-03-31 18:33:21,577 root INFO Squashing layer 'c29e52d44f6950f3cd33faf8d7745a552896b2a64c4c98110f92c23e6c1e0c8b'...
2016-03-31 18:33:21,580 root INFO Squashing layer '40964da1bf475b05d38b509bc53e2304fd405af0fbfb7144157bf9aa18ebb4c8'...
2016-03-31 18:33:21,594 root INFO Squashing layer 'dbcf09c742a7231a963b6e8765c3fe4cd917df10e981e7f224fb4df475258afe'...
2016-03-31 18:33:21,667 root INFO Squashing finished!
2016-03-31 18:33:21,693 root INFO Squashed image name: dev_squashed:latest
2016-03-31 18:33:28,992 root INFO Image registered in Docker daemon as dev_squashed:latest
2016-03-31 18:33:29,196 root INFO Done
Youβll also notice that the layer hashes are different now (both in terms of they are not prefixed with sha256: but also the hash itself is different).
I can rock Docker 1.9 on my build machine for now, but definitely seems like something that should be fixed!
Small note: the image shown in the examples was built with Docker 1.5, but I did test this with an image I built using Docker 1.10 and everything else holds up (I would have demoed that one if apt hadnβt of cleaned it up when I swapped from 1.10 to 1.9 =/).
Issue Analytics
- State:
- Created 7 years ago
- Comments:9 (4 by maintainers)

Top Related StackOverflow Question
Thanks for this report, but I do think this is all fixed already π
There are two things - first - this is a duplicate of #48 which is already fixed. Docker 1.10+ support is available in 1.0.0 pre-releases. Take a look at the releases page. You can install it, for example like this:
pip install -U --user https://github.com/goldmann/docker-scripts/archive/1.0.0rc2.zip. Please use this version and report any bugs with Docker 1.10+.Second thing is the
layerssubcommand not stripping thesha256:part of the layer id. Most probably thelayerscommand will be removed before 1.0.0. Take a look at #70. Instead please usedocker history. Please take a look at release note for 1.0.0rc1 about thelayerssubcommand too πHTH
Yea, I figured this was caused by pip or setuptools and not something you did, but I figured it was worth bringing to your attention!
Opened as issue #77.
Thanks!