Copy-Item incorrect behavior and also inconsistent results when rerun

Steps to reproduce

Issue Copy-Item behavior is inconsistent and produces unexpected results In some cases rerunning the same command produces different results The results are different than what cmd copy, xcopy, and robocopy would produce

SetUp For each example the following setup is to start with a clean slate Before EACH of the tests in EACH example the environment is reset (so before each Copy-Item, cmd copy tests, xcopy, or robocopy test)

SetUp-PSCopyTest.ps1

    rm -r -force \test
    rm -r -force \test2

    md \test

    echo 'a' > \test\a.txt
    echo 'b' > \test\b.txt
    echo 'c' > \test\c.txt
    echo 'd' > \test\d.txt

    md \test\1

    echo '1d' > \test\1\d.txt

Which produces the following folders and files

    \test\a.txt
    \test\b.txt
    \test\c.txt
    \test\d.txt
    \test\1
    \test\1\d.txt

Example 1 - Destination does not exist and recurse not specified Setup SetUp-PSCopyTest.ps1

Test
    Copy-Item test test2

Expected Result
    \test2          directory created
    \test2\a.txt    file copied
    \test2\b.txt    file copied
    \test2\c.txt    file copied
    \test2\d.txt    file copied

Copy-Item Result (UnExpected)
    \test2          directory created but is empty
    No files in \test were copied

Copy-Item ReRun Result (UnExpected AND different)
    \test2\test     directory created
    No files in \test were copied and no files exist in test2 or test2\test

cmd copy test test2 Result (Not desired but expected)
    \test2 FILE created which contains the contents of
        \test\a.txt
        \test\b.txt
        \test\c.txt
        \test\d.txt

    Not the desired result but expected since \test2 directory did not exist

xcopy test test2 Result (Expected)
    The expected results occur if you respond D for directory to the prompt
    asking if test2 is File or Directory

robocopy test test2 Result (Expected)
    The expected results occur

Example 2 - Destination does not exist and recurse was specified Setup SetUp-PSCopyTest.ps1

Test
    Copy-Item test test2 -r

Expected Result
    \test2                  directory created
    \test2\a.txt            file copied
    \test2\b.txt            file copied
    \test2\c.txt            file copied
    \test2\d.txt            file copied
    \test\1                 directory created
    \test\1\d.txt           file copied

Copy-Item Result (Expected)
    The expected results occur

Copy-Item ReRun Result (UnExpected AND different)
    Note when rerun the results are different since the \test2 directory exists
    This makes Copy-Item difficult to use in a script that is scheduled to run
    on a reoccurring basis

    \test2\test             directory created
    \test2\test\a.txt       file copied
    \test2\test\b.txt       file copied
    \test2\test\c.txt       file copied
    \test2\test\d.txt       file copied

    \test2\test\1           directory created
    \test2\test\1\d.txt     file copied

Copy-Item 2nd Rerun Result (UnExpected AND different again from previous 2 runs)
    Error: CopyItem : An item with the specified name \test2\test already exists

    If -Force is specified then the results are the same as the Copy-Item ReRun results,
    however, that demonstrates how Copy-Item cannot be used to produce the same results
    for a rerun command. It also only seems to care when a destination directory
    exists and as other tests show does not care if it would overwrite a file.

cmd copy Result (Not Supported)
    cmd copy does not support recursion so test not applicable

xcopy test test2 /s /e Result (Expected)
    The expected results occur if you respond D for directory to the prompt
    asking if test2 is File or Directory

robocopy test test2 /s /e Result (Expected)
    The expected results occur

Example 3 - Destination exists but is empty and recurse not specified Setup SetUp-PSCopyTest.ps1 md \test2

Test
    Copy-Item test test2

Expected Result
    \test2\a.txt            file copied
    \test2\b.txt            file copied
    \test2\c.txt            file copied
    \test2\d.txt            file copied

Copy-Item Result (UnExpected)
    \test2\test             directory created
    No files in \test were copied

Copy-Item ReRun Result (UnExpected and different)
    Error: copy-item : An item with the specified name \test2\test already exists.

    If -Force is specified then the results are the same as the Copy-Item results

cmd copy test test2 Result (Expected)
    The expected results occur

xcopy test test2 Result (Expected)
    The expected results occur

robocopy test test2 Result (Expected)
    The expected results occur

Example 4 - Destination exists but is empty and recurse was specified Setup SetUp-PSCopyTest.ps1 md \test2

Test
    Copy-Item test test2 -r

Expected Result
    \test2\a.txt            file copied
    \test2\b.txt            file copied
    \test2\c.txt            file copied
    \test2\d.txt            file copied
    \test2\1                directory created
    \test2\1\d.txt          file copied

Copy-Item Result (UnExpected)
    \test2\test\a.txt       file copied
    \test2\test\b.txt       file copied
    \test2\test\c.txt       file copied
    \test2\test\d.txt       file copied
    \test2\test\1           directory created
    \test2\test\1\d.txt     file copied

Copy-Item ReRun Result (UnExpected)
    Error: Copy-Item : An item with the specified name \test2\test already exists.

    If -Force is specified then the results are the same as the Copy-Item results, but
    those are not the expected or desired results

cmd copy Result (Not Supported)
    cmd copy does not support recursion so test not applicable

xcopy test test2 /s /e Result (Expected)
    The expected results occur

robocopy test test2 /s /e Result (Expected)
    The expected results occur

Example 5 - Destination does not exist, using * with the source and recurse not specified Setup SetUp-PSCopyTest.ps1

Test
    Copy-Item test\* test2

Expected Result
    \test2                  directory created
    \test2\a.txt            file copied
    \test2\b.txt            file copied
    \test2\c.txt            file copied
    \test2\d.txt            file copied

Copy-Item Result (Expected)
    The expected results occur

Copy-Item ReRun Result (UnExpected and different)
    \test2\1                directory created and empty

    No confirmation for overwriting files occur

Copy-Item 2nd ReRun Result (UnExpected and different)
    Error: Copy-Item : An item with the specified name \test2\1 already exists.

    Interesting no issue with overwritting the files

    If -Force is specified then the results are the same as the Copy-Item rerun results, but
    those are not the expected or desired results

cmd copy test\* test2 Result (Not desired but expected)
    \test2 FILE created which contains the contents of
        \test\a.txt
        \test\b.txt
        \test\c.txt
        \test\d.txt

    Not the desired result but expected since \test2 directory did not exist

xcopy test\* test2 Result (Expected)
    The expected results occur if you respond D for directory to the prompt
    asking if test2 is File or Directory

robocopy test\* test2 Result (Not supported - but alternative available)
    Robocopy works with directories so this syntax is not valid
    This is not really an issue since robocopy test test2 produces the expected results

Example 6 - Destination does not exist, using * with the source and recurse was specified Setup SetUp-PSCopyTest.ps1

Test
    Copy-Item test\* test2 -r

Expected Result
    \test2                  directory created
    \test2\a.txt            file copied
    \test2\b.txt            file copied
    \test2\c.txt            file copied
    \test2\d.txt            file copied
    \test2\1                directory created
    \test2\1\d.txt          file copied

Copy-Item Result (UnExpected)
    \test2                  directory created
    \test2\a.txt            file copied
    \test2\b.txt            file copied
    \test2\c.txt            file copied
    \test2\d.txt            file copied

    Despite specifying recurse the following were not created
        \test2\1            directory not created
        \test2\1\d.txt      file not copied

Copy-Item ReRun Result (Expected BUT different)
    The expected results occur but this differs from the initial run which shows that
    you cannot run the same command and get the same results.

cmd copy Result (Not Supported)
    cmd copy does not support recursion so test not applicable

xcopy test\* test2 /s /e Result (Expected)
    The expected results occur if you respond D for directory to the prompt
    asking if test2 is File or Directory

robocopy test\* test2 /s /e Result (Not supported - but alternative available)
    Robocopy works with directories so this syntax is not valid
    This is not really an issue since robocopy test test2 /s /e produces the expected results

Example 7 - Destination exists, using * with the source and recurse not specified Setup SetUp-PSCopyTest.ps1 md \test2

Test
    Copy-Item test\* test2

Expected Result
    \test2\a.txt            file copied
    \test2\b.txt            file copied
    \test2\c.txt            file copied
    \test2\d.txt            file copied

Copy-Item Result (UnExpected)
    \test2\a.txt            file copied
    \test2\b.txt            file copied
    \test2\c.txt            file copied
    \test2\d.txt            file copied
    \test2\1                directory created but empty

Copy-Item ReRun Result (UnExpected and different)
    Error: Copy-Item : An item with the specified name \test2\1 already exists.

    Interesting no issue with overwriting the files

    If -Force is specified then the results are the same as the Copy-Item  results, but
    those are not the expected or desired results

cmd copy test\* test2 Result (Expected)
    The expected results occur

xcopy test\* test2 Result (Expected)
    The expected results occur

robocopy test\* test2 Result (Not supported - but alternative available)
    Robocopy works with directories so this syntax is not valid
    This is not really an issue since robocopy test test2 produces the expected results

Example 8 - Destination exists, using * with the source and recurse was specified Setup SetUp-PSCopyTest.ps1 md \test2

Test
    Copy-Item test\* test2 -r

Expected Result
 \test2\a.txt               file copied
 \test2\b.txt               file copied
 \test2\c.txt               file copied
 \test2\d.txt               file copied
 \test2\1                   directory created
 \test2\1\d.txt             file copied

Copy-Item Result (Expected)
    The expected results occur

Copy-Item ReRun Result (Expected If -Force is also used)
    Error: Copy-Item : An item with the specified name \test2\1 already exists.

    If -Force is specifed then the expected results occur

cmd copy Result (Not Supported)
    cmd copy does not support recursion so test not applicable

xcopy test\* test2 /s /e Result (Expected)
    The expected results occur

robocopy test\* test2 /s /e Result (Not supported - but alternative available)
    Robocopy works with directories so this syntax is not valid
    This is not really an issue since robocopy test test2 /s /e produces the expected results

Example 9 - Destination does not exist, using . with the source and recurse not specified Setup SetUp-PSCopyTest.ps1

Test
    Copy-Item test\*.* test2

Expected Result
 \test2                     directory created
 \test2\a.txt               file copied
 \test2\b.txt               file copied
 \test2\c.txt               file copied
 \test2\d.txt               file copied

Copy-Item Result (UnExpected but similar to what cmd copy did in prior test)
    test2 FILE created with the contents of \test\d.txt only

Copy-Item ReRun Result (UnExpected)
    Same results as Copy-Item Result, however, interesting that no confirmation prompt
    displayed to overwrite the file

cmd copy test\*.* test2 Result (Not desired but expected)
    \test2 FILE created which contains the contents of
        \test\a.txt
        \test\b.txt
        \test\c.txt
        \test\d.txt

xcopy test\*.* test2 Result (Expected)
    The expected results occur if you respond D for directory to the prompt
    asking if test2 is File or Directory

robocopy test\*.* test2 Result (Not supported - but alternative available)
    Robocopy works with directories so this syntax is not valid
    This is not really an issue since robocopy test test2 produces the expected results

Example 10 - Destination does not exist, using . with the source and recurse was specified Setup SetUp-PSCopyTest.ps1

Test
    Copy-Item test\*.* test2 -r

Expected Result
    \test2                  directory created
    \test2\a.txt            file copied
    \test2\b.txt            file copied
    \test2\c.txt            file copied
    \test2\d.txt            file copied
    \test2\1                directory created
    \test2\1\d.txt          file copied

Copy-Item Result (UnExpected but similar to what cmd copy did in prior test)
    \test2 FILE created which contains the contents of \test\d.txt

Copy-Item ReRun Result (UnExpected)
    Same as Copy-Item Result but no overwrite prompt

cmd copy Result (Not Supported)
    cmd copy does not support recursion so test not applicable

xcopy test\*.* test2 /s /e Result (Expected)
    The expected results occur if you respond D for directory to the prompt
    asking if test2 is File or Directory

robocopy test\*.* test2 /s /e Result (Not supported - but alternative available)
    Robocopy works with directories so this syntax is not valid
    This is not really an issue since robocopy test test2 /s /e produces the expected results

Example 11 - Destination exists, using . with the source and recurse not specified Setup SetUp-PSCopyTest.ps1 md \test2

Test
    Copy-Item test\*.* test2

Expected Result
    \test2\a.txt            file copied
    \test2\b.txt            file copied
    \test2\c.txt            file copied
    \test2\d.txt            file copied

Copy-Item Result (Expected)
    The expected results occur

Copy-Item ReRun Result (Expected)
    The expected results occur, however, there is no warning about overwritting files

cmd copy test\*.* test2 Result (Expected)
    The expected results occur

xcopy test\*.* test2 Result (Expected)
    The expected results occur

robocopy test\*.* test2 Result (Not supported - but alternative available)
    Robocopy works with directories so this syntax is not valid
    This is not really an issue since robocopy test test2 produces the expected results

Example 12 - Destination exists, using . with the source and recurse was specified Setup SetUp-PSCopyTest.ps1 md \test2

Test
    Copy-Item test\*.* test2 -r

Expected Result
\test2\a.txt                file copied
\test2\b.txt                file copied
\test2\c.txt                file copied
\test2\d.txt                file copied
\test2\1                    directory created
\test2\1\d.txt              file copied

Copy-Item Result (UnExpected)
    \test2\a.txt            file copied
    \test2\b.txt            file copied
    \test2\c.txt            file copied
    \test2\d.txt            file copied

    Despite specifying recurse the following were not created
        \test2\1            directory not created
        \test2\1\d.txt      file not copied

Copy-Item ReRun Result (UnExpected)
    Same results as Copy-Item, however, there is no warning about overwritting files

    Despite specifying recurse the following were not created
        \test2\1            directory not created
        \test2\1\d.txt      file not copied

cmd copy Result (Not Supported)
    cmd copy does not support recursion so test not applicable

xcopy test\*.* test2 /s /e Result (Expected)
    The expected results occur

robocopy test\*.* test2 /s /e Result (Not supported - but alternative available)
    Robocopy works with directories so this syntax is not valid
    This is not really an issue since robocopy test test2 /s /e produces the expected results

– – Comparison of copy method results

Copy-Item
    1   UnExpected result and different results when rerun
    2   Expected result   but different results when rerun
    3   UnExpected result and different results when rerun
    4   UnExpected result and different results when rerun
    5   Expected result   but different results when rerun
    6   UnExpected result initially and Expected Result when rerun
    7   UnExpected result and different results when rerun
    8   Expected result   and same result when rerun if -Force used
    9   UnExpected result and same result when rerun
    10  UnExpected result and same result when rerun
    11  Expected result   and same result when rerun
    12  UnExpected result and same result when rerun

cmd copy
    1   Not desired but expected result
    2   Does not support recursion
    3   Expected result
    4   Does not support recursion
    5   Not desired but expected result
    6   Does not support recursion
    7   Expected result
    8   Does not support recursion
    9   Not desired but expected result
    10  Does not support recursion
    11  Expected result
    12  Does not support recursion

xcopy
    1   Expected result
    2   Expected result
    3   Expected result
    4   Expected result
    5   Expected result
    6   Expected result
    7   Expected result
    8   Expected result
    9   Expected result
    10  Expected result
    11  Expected result
    12  Expected result

robocopy
    1   Expected result
    2   Expected result
    3   Expected result
    4   Expected result
    5   Not Supported but alternative syntax works
    6   Not Supported but alternative syntax works
    7   Not Supported but alternative syntax works
    8   Not Supported but alternative syntax works
    9   Not Supported but alternative syntax works
    10  Not Supported but alternative syntax works
    11  Not Supported but alternative syntax works
    12  Not Supported but alternative syntax works

– – Conclusion

When I first started using PowerShell back in the 2.0 days I noticed these and similar issues but figured they would be addressed as the product matured, therefore I reverted to using cmd copy, xcopy, and robocopy, however, I wanted to use Copy-Item since in theory it provided the capabilites of cmd copy and xcopy all in one.

It is disappointing to see that we have version 7.0 out now in beta and they still have not been addressed basic functionality.

Copy-Item produces the expected results only 33% of the time and if rerun then it only produces the same results 16% of the time.

If it was just a matter of learning a different syntax that would be one thing, however, when considering common use cases (destination exists, destination does not exist, recurse the source directory tree, don’t recurse the source directory tree), there does not appear to be a syntax which handles all of those cases AND can produce the same results when rerun.

This makes Copy-Item pretty much unusable excpet for all but the most simple types of copy operations.

Comparing Copy-Item to the other options, cmd copy generally works (with the exception of the not supporting recursive and a few other cases), xcopy always works, and robocopy always works OR works with a tweak to the syntax.

Back in the PS 2.0 days I considered writing my own Copy-Item replacement but wanted to use built in functionality as much as possible so I did not do that.

Are there any plans to address these issues?

There seem to be several key issues:

1. When the destination does not exist and you are copying more than one item it should prompt to ask if the destination should or not the destination should be a container. That covers both use cases where you want to copy multiple items to a single item or where you want to copy multiple items to a container.

2. Recurse needs to be fixed so that when you specify recurse it recurses and when you don’t specify recurse it does not create empty containers/folders.

3. Why does it produce an error if a destination directory exists but not when a destination file exists? Based on PowerShell guidelines since overwritting a file is a destructive operation there should be a confirmation before doing so. Having the user change their $ConfirmationPreference from the default of ‘High’ is not the solution here because that would cause all sorts of other confirmations that do not currently occur. Since this constitutes destructive behavior the cmdlet should be changed to confirm.

I’m sure there are other examples, but those seem to be the key areas causing the problems.

This basic functionality in the product really needs to be fixed so that we do not have to restore to using external tools to accomplish these basic tasks.

All the above was reproduced on PS 7 Preview 3

Expected behavior

Actual behavior

Environment data

Name Value

---

PSVersion 7.0.0-preview.3 PSEdition Core GitCommitId 7.0.0-preview.3 OS Microsoft Windows 10.0.18362 Platform Win32NT PSCompatibleVersions {1.0, 2.0, 3.0, 4.0…} PSRemotingProtocolVersion 2.3 SerializationVersion 1.1.0.1 WSManStackVersion 3.0

[JoeSalmeri]

@SteveL-MSFT

Sorry, I did not mean to imply that the problem was your doing or something you created and I certainly do appreciate all you have done since joining the team!

My issue is with the answers provided regarding the problem.

I have been in the field for 32 years and have programmed in at least 25 different programming languages during my career.

In all cases there is one common theme and answer to the question, “Why do we write programs?”

We program so that we can automate processes and produce reliable, consistent, and repeatable results.

There are very few cases or reason why we would want to write code which produced results which were not repeatable or consistent (a random number generator comes to mind) as that would defeat the purpose of automating tasks.

If we were discussing what the expected behavior was versus what the actual behavior was that is something that could be addressed (but maybe not desired) by changing your/my expectations to match the actual behavior.

That is NOT the case here. The same code run multiple times does not produce the same results.

Before taking the time to write up that bug and all the various situations, I spent CONSIDERABLE time trying various syntax (trailing slashes, *, ., and others) to try and find a single syntax that would produce a consistent result and did not find any for the uses cases described.

A good developer is not going to write new code using existing code which does not produce repeatable results because then their code also becomes unreliable.

I’ll bet that most PS developers have taken the route that I have which is to use external tools to perform that functionality in their code so that they can get consistent repeatable results.

I take breaking code issues quite seriously too, however, when the original code is broke in the first place, it makes sense to fix it regardless of whether it would break “other” code.

Imagine if the existing Copy-Item had a bug where it deleted the file after it copied it. Using the answer you provided, that would not be addressed because it might break existing code that depended on that bug. I cannot imagine anyone considering that a good reason or answer.

It’s been a while since I wrote all that up and reviewed it, however, there were also examples where data loss occurs because there was no warning or confirmation. That certainly goes against the basic PS design principles of not doing harm without a warning.

While I appreciate that you opened the issue for FileSystemProvider V2, I am quite skeptical that it will ever occur based on your response here, your comment about “if #5785 ever gets done”, and the fact that these issues have gone unaddressed pretty much since the beginning of PS.

Regardless of whether the existing code is fixed or whether a V2 is created, there should still be a desire to provide this basic functionality in the product, especially one that is marketed as the replacement for all of the other shells.

I really hope that someone take up the task of addressing this.

[iSazonov]

@JoeSalmeri Thanks for the in-depth report! It would be easier for us to move forward if you converted your examples into Pester tests with comments and questions so that we could discuss each one and draw a conclusion. Perhaps we’ll need to split this topic into several to facilitate the creation of fixes. Also remember that we should consider all providers, not only FileSystem, and avoid breaking changes.