Blob feed publish failure, DirectoryNotFoundException for Sleet temp file

Failure in Core-Setup release/3.0: https://dev.azure.com/dnceng/internal/_build/results?buildId=101999

  Waiting to obtain an exclusive lock on the feed.
  GET https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json
  System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\Users\vsagent\AppData\Local\Temp\a431713f-c412-48fe-8de6-d6cad755e912\3bb3eaa0-b744-4a52-9138-232868f2ff76.tmp'.
     at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
     at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
     at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
     at Sleet.AzureFile.<CopyFromSource>d__2.MoveNext()
  --- End of stack trace from previous location where exception was thrown ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
     at Sleet.FileBase.<EnsureFile>d__37.MoveNext()
  Failed to sync 'https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json'. Retrying.
  GET https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json
  System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\Users\vsagent\AppData\Local\Temp\a431713f-c412-48fe-8de6-d6cad755e912\3bb3eaa0-b744-4a52-9138-232868f2ff76.tmp'.
     at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
     at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
     at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
     at Sleet.AzureFile.<CopyFromSource>d__2.MoveNext()
  --- End of stack trace from previous location where exception was thrown ---
     at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
     at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
     at Sleet.FileBase.<EnsureFile>d__37.MoveNext()
  Failed to sync 'https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json'. Retrying.
...
  GET https://dotnetfeed.blob.core.windows.net/dotnet-core/index.json
##[error]publish\publish.proj(288,5): Error : Could not find a part of the path 'C:\Users\vsagent\AppData\Local\Temp\a431713f-c412-48fe-8de6-d6cad755e912\3bb3eaa0-b744-4a52-9138-232868f2ff76.tmp'.
F:\vsagent\11\s\publish\publish.proj(288,5): error : Could not find a part of the path 'C:\Users\vsagent\AppData\Local\Temp\a431713f-c412-48fe-8de6-d6cad755e912\3bb3eaa0-b744-4a52-9138-232868f2ff76.tmp'.

It waited for the lock for 1h 37m. Is it possible Sleet set up a temp file that was deleted in this timeframe? Or is there something in Arcade that might be fixable?

I’ve only seen this once so far, so it might be just another way central feed publish can be flaky.

@markwilkie

[emgarten]

I’m more thinking that some cleanup process on the build agent might have deleted the temp file, not Sleet or the build task.

This would be my guess also, sleet doesn’t do anything while waiting for the lock since the feed will change before it is unlocked.

maybe we can change where Sleet’s temp files go to avoid it.

I’ve considered adding an option for this before, it could be done.

[dagood]

I’m more thinking that some cleanup process on the build agent might have deleted the temp file, not Sleet or the build task. If so (and if this actually repros enough to make it worth it, which I doubt) maybe we can change where Sleet’s temp files go to avoid it.