Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
dvc push: fails on 2.4.1 with TypeError
See original GitHub issueBug Report
Description
When running dvc push, a TypeError occurs and the push fails. Does not occur with all dvc push commands, only for a certain stage.
I’ve confirmed this does not occur on DVC version 2.1.0. I haven’t tried other versions.
dvc push -v
2021-06-28 08:14:45,050 DEBUG: Check for update is enabled.
2021-06-28 08:14:45,106 DEBUG: Creating external repo git@gitlab.example.com:root/dataset-registry.git@94955872fe7ec72e24b634246a58aa71f49891e9
2021-06-28 08:14:45,106 DEBUG: erepo: git clone 'git@gitlab.example.com:root/dataset-registry.git' to a temporary dir
2021-06-28 08:15:13,698 DEBUG: Checking if stage '/tmp/tmp6lare_isdvc-clone/gesture/trinity.tar.gz' is in 'dvc.yaml'
2021-06-28 08:15:14,214 DEBUG: Preparing to upload data to 's3://datasets'
2021-06-28 08:15:14,214 DEBUG: Preparing to collect status from s3://datasets
2021-06-28 08:15:14,214 DEBUG: Collecting information from local cache...
2021-06-28 08:15:14,217 DEBUG: Collecting information from remote cache...
2021-06-28 08:15:14,217 DEBUG: Querying 5 hashes via object_exists
2021-06-28 08:15:14,327 DEBUG: Querying 1 hashes via object_exists
2021-06-28 08:15:14,356 DEBUG: Matched '0' indexed hashes
2021-06-28 08:15:15,143 DEBUG: '00' doesn't look like a cache file, skipping
2021-06-28 08:15:15,144 DEBUG: Estimated remote size: 4096 files
2021-06-28 08:15:15,145 DEBUG: Querying '21' hashes via traverse
2021-06-28 08:15:15,162 ERROR: unexpected error - can only concatenate str (not "NoneType") to str
------------------------------------------------------------
Traceback (most recent call last):
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/main.py", line 55, in main
ret = cmd.do_run()
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/command/base.py", line 50, in do_run
return self.run()
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/command/data_sync.py", line 57, in run
processed_files_count = self.repo.push(
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/repo/__init__.py", line 51, in wrapper
return f(repo, *args, **kwargs)
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/repo/push.py", line 44, in push
pushed += self.cloud.push(objs, jobs, remote=remote)
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/data_cloud.py", line 79, in push
return remote_obj.push(
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/remote/base.py", line 57, in wrapper
return f(obj, *args, **kwargs)
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/remote/base.py", line 494, in push
ret = self._process(
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/remote/base.py", line 351, in _process
dir_status, file_status, dir_contents = self._status(
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/remote/base.py", line 195, in _status
self.hashes_exist(
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/remote/base.py", line 145, in hashes_exist
return indexed_hashes + self.odb.hashes_exist(list(hashes), **kwargs)
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/objects/db/base.py", line 467, in hashes_exist
remote_hashes = set(
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/objects/db/base.py", line 312, in list_hashes_traverse
yield from itertools.chain.from_iterable(in_remote)
File "/home/example/anaconda3/envs/moglow/lib/python3.9/concurrent/futures/_base.py", line 608, in result_iterator
yield fs.pop().result()
File "/home/example/anaconda3/envs/moglow/lib/python3.9/concurrent/futures/_base.py", line 445, in result
return self.__get_result()
File "/home/example/anaconda3/envs/moglow/lib/python3.9/concurrent/futures/_base.py", line 390, in __get_result
raise self._exception
File "/home/example/anaconda3/envs/moglow/lib/python3.9/concurrent/futures/thread.py", line 52, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/objects/db/base.py", line 302, in list_with_update
return list(
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/objects/db/base.py", line 193, in list_hashes
for path in self._list_paths(prefix, progress_callback):
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/objects/db/base.py", line 173, in _list_paths
for file_info in self.fs.walk_files(path_info, prefix=prefix):
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/fs/fsspec_wrapper.py", line 109, in walk_files
for file in self.find(path_info, **kwargs):
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/dvc/fs/fsspec_wrapper.py", line 178, in find
files = self.fs.find(
File "/home/example/.local/lib/python3.9/site-packages/fsspec/asyn.py", line 87, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "/home/example/.local/lib/python3.9/site-packages/fsspec/asyn.py", line 68, in sync
raise result[0]
File "/home/example/.local/lib/python3.9/site-packages/fsspec/asyn.py", line 24, in _runner
result[0] = await coro
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/s3fs/core.py", line 602, in _find
out = await self._lsdir(path, delimiter="", prefix=prefix)
File "/home/example/anaconda3/envs/moglow/lib/python3.9/site-packages/s3fs/core.py", line 517, in _lsdir
prefix = key.lstrip("/") + "/" + prefix
TypeError: can only concatenate str (not "NoneType") to str
------------------------------------------------------------
2021-06-28 08:15:15,214 DEBUG: Version info for developers:
DVC version: 2.4.1 (pip)
---------------------------------
Platform: Python 3.9.5 on Linux-5.11.0-22-generic-x86_64-with-glibc2.33
Supports: http, https, s3
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/sda3
Caches: local
Remotes: s3
Workspace directory: ext4 on /dev/sda3
Repo: dvc, git
Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2021-06-28 08:15:15,215 DEBUG: Analytics is enabled.
2021-06-28 08:15:15,288 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmpgspzaj7d']'
2021-06-28 08:15:15,289 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmpgspzaj7d']'
Reproduce
- dvc run -n … -d … -o … “…command…”
- dvc push
Expected
DVC should push tracked content to S3 repo.
Environment information
DVC version: 2.4.1 (pip)
---------------------------------
Platform: Python 3.9.5 on Linux-5.11.0-22-generic-x86_64-with-glibc2.33
Supports: http, https, s3
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/sda3
Caches: local
Remotes: s3
Workspace directory: ext4 on /dev/sda3
Repo: dvc, git```
Issue Analytics
- State:
- Created 2 years ago
- Comments:11 (5 by maintainers)
Top Results From Across the Web
dir file not pushed because of progress bar error · Issue #7704
This causes dvc to think that the file upload failed and so it decides not to upload the .dir file. Actually the file...
Read more >Troubleshooting | Data Version Control - DVC
Failed to pull data from the cloud The most common cause is changes pushed to Git without the corresponding data being uploaded to...
Read more >How to fix DVC error 'FileNotFoundError: [Errno 2] No such file ...
It sounds like the file doesn't exist in the remote. It should be dvc push ed first from wherever it originated. · I...
Read more >Functional Programming and Embedded Systems
Phil Wadler inspired the push to convert to monadic 1/0 ... Any other message to or from that process should constitute a static...
Read more >mon-doc-help-CL.el - EmacsWiki
Push each lisp list containing the text-properties needed for facificaton, ... `type-error-datum' `package-error-package' `type-error-expected-type' ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I’ve updated and it looks like it works. Thanks!!
I downgraded, pushed properly, went back to 2.4.1 and it doesn’t happen anymore. But this was the error I was getting: