dvc push: it's run successfully but the remote path has no file
See original GitHub issueBug Report
Description
I setup a hdfs path as remote storage, and dvc add and then push a file successfully according to the terminal message. But I can’t see files in the hdfs path.
Reproduce
0, dvc init 1, dvc add data.csv 2, dvc remote add -d myhdfs hdfs://xxx/yyy 3, dvc push 4, hadoop fs -ls hdfs://xxx/yyy
Expected
Expect to see some files in the hdfs path after pushing successfully, but got nothing there.
Environment information
Output of hadoop fs -ls hdfs://xxx/yyy
:
$
Additional Information (if any): After runing dvc push, the output is
$ 0% Querying cache in yyy|
22/05/24 20:50:26 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
22/05/24 20:50:26 WARN shortcircuit.DomainSocketFactory: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
Everything is up to date.
Issue Analytics
- State:
- Created a year ago
- Comments:7 (3 by maintainers)
Top Results From Across the Web
Troubleshooting | Data Version Control - DVC
The most common cause is changes pushed to Git without the corresponding data being uploaded to the DVC remote. Make sure to dvc...
Read more >dvc push: when not all files are in local cache, still ... - GitHub
The issue was pushing incomplete datasets from my gpu server to the new storage remotes. It pushed all the files present in the...
Read more >Data & Model Management with DVC | Analytics Vidhya
In this post we learn about versioning for ML projects & use DVC to version & maintain ML artifacts in a remote Amazon...
Read more >Cannot add 'folder-path', because it is overlapping with other ...
dvc files in my project. $ dvc add ./project_model/data/ ERROR: Cannot add '/home/me/PycharmProjects/project/project_model/data/images' ...
Read more >November '19 DVC❤️Heartbeat - Iterative.ai
Git commit id (hash) is saved in DVC file upon import, data itself is copied/downloaded into DVC repo cache but would not be...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@daavoo I tried 2.10.2, and it worked. I can push to and find the files in my remote path.
Could you try with
2.10.2
? I think this was solved in https://github.com/iterative/dvc/pull/7563