Failure message: cp skipping file, as it was replaced while being copied
See original GitHub issueWorkflow that I’m testing:
- Mount a google storage bucket with gcsfuse
- Copy a binary from that bucket to the current working directory
- Make that binary executable
- Run that binary
However, when I’m doing this for multiple simultaneous jobs, I occasionally get error messages like the following:
command--james--190429-094127-78.6 (attempt 1) failed. Retrying.
Failure message: cp: skipping file '/mnt/data/mount/gs/my_bucket/projects/jamesp/bin/my-executable', as it was replaced while being copied
One possibility is that my working directory is within the mounted bucket, in which case copying the file to itself (by another simultaneous job) could trigger that. But that doesn’t seem like what I’d expect, just from mounting a bucket. To clarify, what is the current working directory by default when a job is launched in dsub?
Issue Analytics
- State:
- Created 4 years ago
- Comments:9 (2 by maintainers)
Top Results From Across the Web
"cp: skipping file 'file.txt', as it was replaced while being copied"
I am getting the (apparently) classic error: "cp: skipping file 'file.txt', as it was replaced while being copied" It only occurs on a...
Read more >cp: skipping file x' as it was replaced while being copied ...
While copying files from another server (cifs) I get the following error: cp: skipping file `xxxxxxx', as it was replaced while being copied....
Read more >cp or mv: skipping file...it was replaced - LinuxQuestions.org
Files are copied from the development samba share mounted on the production box to the local filesystem on the production box. However, when...
Read more >Cannot move file in shared folder after upgrade to PD16
Error is "cp: skipping file './test.cpp', as it was replaced while being copied". The Guest OS is Centos 7.5.1804. Attached Files ...
Read more >Re: cp: skipping file 'file', as it was replaced while being copied
Re: cp: skipping file 'file', as it was replaced while being copied · From: Linda Walsh <cygwin at tlinx dot org> · To:...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
If you are able to create a reproducible test case that demonstrates clearly that the file is in fact not being modified while you are trying to copy it, then it seems worth filing an issue with gcsfuse.
I’m going to leave this issue open for now as I think we should update the
dsub
documentation to more clearly indicate that using the bucket--mount
flag should be done only when you’ve really proved out that the standard--input
/--output
mechanisms are insufficient.Thanks.
you must recreate your source file from another gcsfuse.
this error actually because gcsfuse has a “cache” about a file, this cache has a default timeout (1s), when you
cp file1 file2
, cp command first statfile1
file, that maybe hit the cache, and then open file1 file, then stat file1 file again, then cache maybe timeout, this stat will get file1’s info from backend storage, cp command then compare two stat’s result, if it’s not same, that means, file1 maybe recreated, then cp command will return error message " skipping file, as it was replaced while being copied".