Add-Content and Out-File -Append can fail quietly, seemingly connected to cross-thread concurrency

Note:

• I don’t know what other cmdlets are affected. Update: at least Out-File -Append is affected too.

• A common use case where the problem can surface is in the context of ForEach-Object -Parallel (while the use of Add-Content in parallel threads requires explicit synchronization, neglecting to do so should at least report the failure to write.)

Steps to reproduce

$tempFile = New-TemporaryFile
Write-Verbose -vb "Waiting for an Add-Content call to fail..."
$Error.Clear()
while ($true) {
  Remove-Item -ea Ignore $tempFile
  # Let multiple threads try to update Add-Content simultaneously.
  1..100 | ForEach-Object -Parallel { Add-Content $using:tempFile -Value $_ -ErrorAction Stop; if (-not $?) { Write-Warning $_.ToString() } }
  # On occasion, the file ends up with fewer than 100 lines.
  $countWritten = (Get-Content $tempFile).Count
  if ($countWritten -ne 100) {
    Write-Verbose -vb "$countWritten lines were written."
    # If fewer lines were written, an error should have been reported.
    $Error.Count | Should -BeGreaterOrEqual 1
  }
}
Remove-Item $tempFile

Expected behavior

The tests should run indefinitely, because the Add-Content calls should either all succeed or those that fail should report an error.

Actual behavior

A test eventually (may take a while) fails, because an Add-Content calls fails silently; e.g.:

VERBOSE: 99 lines were written.
InvalidResult:
Line |
  13 | $Error.Count | Should -BeGreaterOrEqual 1
     | Expected the actual value to be greater than or equal to 1, but got 0.

Environment data

PowerShell Core 7.2.0-preview.1

[PaulHigin]

Yes, PowerShell cmdlets are not and were never intended to be multi-threaded safe. Users must beware of using cmdlets in multiple threads, whether via foreach -parallel or PowerShell API. Same as with any non-multi-threaded-safe .NET object.

[iSazonov]

Now .Net 6.0 has get new thread-safe file IO API. https://devblogs.microsoft.com/dotnet/file-io-improvements-in-dotnet-6/#thread-safe-file-io

but I doubt that we will be able to use this.

<div> .NET Blog</div><div>File IO improvements in .NET 6</div><div>Learn about high-performance file IO features in NET 6, like concurrent reads and writes, scatter/gather IO and many more.</div>