Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

push/pull/status/fetch: improve progress bar

See original GitHub issue

Ubuntu: 18.04 dvc: 0.22.0

I added dvc later to an already existing project like so:

dvc add data
dvc run -d data -d train.py -o model -f train.dvc --no-exec train.py ...

(where data and model are directories with lots of image-files resp. big ML-models)

I’ve added a remote S3, commited dvc-files into git and when calling: dvc push dvc hangs at “20% Collecting information” Calling dvc push -v provides lots of

Debug: SELECT * from state WHERE inode=1576004
Debug: fetched: [(1576004, '1544624726428000000', '449ec1c0d6beeadb55b6b4808ce41ec7', '1544630970012086784')]
Debug: Inode '1576004', mtime '1544624726428000000', actual mtime '1544624726428000000'.
Debug: UPDATE state SET timestamp = "1544630972341281280" WHERE inode = 1576004

... and the last log-output is:

Debug: File '/home/blabla/.dvc/cache/44/9ec1c0d6beeadb55b6b4808ce41ec7', md5 '449ec1c0d6beeadb55b6b4808ce41ec7', actual '449ec1c0d6beeadb55b6b4808ce41ec7'
Debug: Path /home/blabla/.dvc/cache/07/270488711c768df5ad02185cb81493 inode 1576007
Debug: SELECT * from state WHERE inode=1576007
Debug: fetched: [(1576007, '1544624727832000000', '07270488711c768df5ad02185cb81493', '1544630970013530368')]
Debug: Inode '1576007', mtime '1544624727832000000', actual mtime '1544624727832000000'.
Debug: UPDATE state SET timestamp = "1544630972343799552" WHERE inode = 1576007
Debug: File '/home/blabla/.dvc/cache/07/270488711c768df5ad02185cb81493', md5 '07270488711c768df5ad02185cb81493', actual '07270488711c768df5ad02185cb81493'
Debug: File '/home/blabla/.dvc/cache/ea/ad01785397a0a43a25cf21fb816b8e', md5 'eaad01785397a0a43a25cf21fb816b8e', actual 'None'
[######                        ] 20% Collecting information

What’s going wrong here?

Issue Analytics

State:
Created 5 years ago
Reactions:1
Comments:5 (3 by maintainers)

Top GitHub Comments

2reactions

mwalmsleycommented, Dec 14, 2018

I had the same problem: paused for 2h+.

One workaround for now (thanks to Ruslan on Discord): specify no-cache, and then compress to a single file (which dvc is then happy with), push, and decompress.

dvc run -d small_stuff -O image_dir python make_images_dir.py 
dvc run -d image_dir -o images.tar tar -cf images.tar image_dir

tar still takes a while, of course, but it eventually works.

1reaction

efiopcommented, Dec 12, 2018

@stvogel Glad it worked eventually 😃 Let’s keep this open until we at least improve information messages(e.g. might add an additional progress bar or just modify the current one to be more informative).