dotnet build fails on .sln with IOException

Recently, and all of a sudden, we’re consistently running into a race condition in our builds. Initially I believed this was caused by the fact we build passing the .sln file and were also passing a runtime identifier (rid), but even without that the problem still occurs quite regularly.

I’ve deleted my comments off https://github.com/dotnet/sdk/issues/863 as it’s no longer an issue with just specifying the rid, something else is going on. I’ve re-iterated them in here.

Scenario A docker container build. I’m specifically building a base image used by all our individual microservice images to speed things up. As such, restoring and building by pointing at the .sln file makes most sense. (It’s not an exact science, but… that’s Docker builds…)

My builder Dockerfile is doing:

# Restore NuGet packages
RUN dotnet restore Microservices.sln    # not doing this anymore: -r linux-musl-x64

# Copy across the rest of the source files
WORKDIR ..
COPY ./ ./

# Build entire sln
RUN dotnet build Microservices/Microservices.sln -c Release --no-restore # not doing this anymore: -r linux-musl-x64

My services Dockerfile is doing this:

# Test
RUN dotnet test ../Tests/${SERVICE_NAME}.Tests/${SERVICE_NAME}.Tests.csproj -c Release --no-restore

# Publish
RUN dotnet publish -c Release --no-restore -o /dist # not doing this anymore: -r linux-musl-x64

My docker-compose.yml is essentially specifying that each service builds itself based on the common microservices-builder image:

version: '3.7'

services:
  microservices-builder:
    image: microservices-builder
    build:
      context: ../..
      dockerfile: Microservices/Docker/builder/Dockerfile

  booking.service:
    image: booking.service
    depends_on:
      - "microservices-builder"
    build:
      context: ..
      dockerfile: BookingService/Dockerfile
    ports:
     - "33000:80"

  rating.service:
    ... etc

Expected Build succeeds.

Actual Build fails due to a IOException (used by another process) on a shared library deps.json file. The specific step is the RUN dotnet build ... in the builder Dockerfile.

10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018: The "GenerateDepsFile" task failed unexpectedly. [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018: System.IO.IOException: The process cannot access the file '/src/AutoGuru.Shared.Utilities/src/bin/Release/netstandard2.0/AutoGuru.Shared.Utilities.deps.json' because it is being used by another process. [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018:    at System.IO.FileStream.Init(FileMode mode, FileShare share, String originalPath) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018:    at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018:    at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018:    at System.IO.File.Create(String path) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018:    at Microsoft.NET.Build.Tasks.GenerateDepsFile.WriteDepsFile(String depsFilePath) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018:    at Microsoft.NET.Build.Tasks.GenerateDepsFile.ExecuteCore() [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018:    at Microsoft.NET.Build.Tasks.TaskBase.Execute() [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]

What I’ve tried

1. No longer specify the rid in the dotnet cli commands, instead specify it in the Directory.Build.props file conditionally when I know a docker build is happening. (initially was doing this only on project files which ended in Service.csproj, but removed that and still the same)

  <PropertyGroup>
    <RuntimeIdentifier Condition="'$(AUTOGURU_IS_DOCKER_BUILD)' == 'True'">linux-musl-x64</RuntimeIdentifier>
  </PropertyGroup>

1. Uncheck “Build” for the .dcproj in Release configuration, just so that’s not doing anything as it doesn’t need to.
2. Attempt to disable a parallel build by passing --disable-parallel to dotnet build.

Even though the Dockerfile for my services is doing a specific dotnet test and a dotnet publish in the specific project’s directory, the intent is really that these don’t need to do much (–no-restore is passed, etc.) as the image it’s based on from the first Dockerfile has done all the hard work and cached everything along the way. This is essential in docker builds to speed up things.

Any help would be greatly appreciated.

[cw-pratik]

I had the same issue on ubuntu. I have installed donet sdk 3.1.401.

I tried a lot of things like doing a restore separately and calling the build-server shutdown command. Nothing worked.

Then I tried the suggestion of @rainersigwald and set -maxcpucount=1 flag and ran it. It worked.

I looked at my project structure and there are 7 projects in the solution all referencing other projects in a tree like manner. But there were additional references like this.

A.csproj -- references B,C,D
| = B.csproj -- references C,D
| | = C.csproj -- references D
| | | = D.csproj

I updated the references in such a way that trainsient references are not stated in the csproj file.

A.csproj -- references B
| = B.csproj -- references C
| | = C.csproj -- references D
| | | = D.csproj

And this fixed the issue.

Though I would have thought that the build command would handle that.

Also no idea what would happen in the following scenario

A.csproj -- references B
| = B.csproj -- references C,E
| | = C.csproj -- references D
| | = E.csporj -- references D
| | | = D.csproj 

Here project D is references by 2 independant projects. This will most likely give the same error as above.

[benmccallum]

FYI, @wli3’s suggestion to shutdown the build server appears to have not worked. Just saw this on TeamCity (on our current dotnetcore 2.2 branch). Again the culprit is AutoGuru.Shared.Utilities.csproj. Interesting. I’ll definitely try the cpu max count switch if it keeps causing me issues.

[00:51:31][Step 2/10] Step 19/21 : RUN dotnet build-server shutdown --msbuild --vbcscompiler --razor &&     dotnet build Microservices/AutoGuru.Microservices.sln -c Release --no-restore
[00:51:31][Step 2/10]  ---> Running in f6a200376132
[00:51:31][Step 2/10] Docker event: {"status":"create","id":"f6a2003761325365a607718151916292b83667097c3c176aacb36c0037b19efa","from":"sha256:572f2c8d3e7a3bc157258a128fee9a699fd1edff11df88e2621094c2476a66d8","Type":"container","Action":"create","Actor":{"ID":"f6a2003761325365a607718151916292b83667097c3c176aacb36c0037b19efa","Attributes":{"image":"sha256:572f2c8d3e7a3bc157258a128fee9a699fd1edff11df88e2621094c2476a66d8","name":"vigilant_mahavira"}},"scope":"local","time":1587739891,"timeNano":1587739891462893775}
[00:51:31][Step 2/10] Docker event: {"status":"attach","id":"f6a2003761325365a607718151916292b83667097c3c176aacb36c0037b19efa","from":"sha256:572f2c8d3e7a3bc157258a128fee9a699fd1edff11df88e2621094c2476a66d8","Type":"container","Action":"attach","Actor":{"ID":"f6a2003761325365a607718151916292b83667097c3c176aacb36c0037b19efa","Attributes":{"image":"sha256:572f2c8d3e7a3bc157258a128fee9a699fd1edff11df88e2621094c2476a66d8","name":"vigilant_mahavira"}},"scope":"local","time":1587739891,"timeNano":1587739891463057063}
[00:51:31][Step 2/10] Docker event: {"status":"start","id":"f6a2003761325365a607718151916292b83667097c3c176aacb36c0037b19efa","from":"sha256:572f2c8d3e7a3bc157258a128fee9a699fd1edff11df88e2621094c2476a66d8","Type":"container","Action":"start","Actor":{"ID":"f6a2003761325365a607718151916292b83667097c3c176aacb36c0037b19efa","Attributes":{"image":"sha256:572f2c8d3e7a3bc157258a128fee9a699fd1edff11df88e2621094c2476a66d8","name":"vigilant_mahavira"}},"scope":"local","time":1587739891,"timeNano":1587739891885039400}
[00:51:32][Step 2/10] Shutting down MSBuild server...
[00:51:32][Step 2/10] Shutting down VB/C# compiler server...
[00:51:32][Step 2/10] MSBuild server shut down successfully.
[00:51:32][Step 2/10] VB/C# compiler server shut down successfully.
[00:51:33][Step 2/10] Microsoft (R) Build Engine version 16.0.450+ga8dc7f1d34 for .NET Core
[00:51:33][Step 2/10] Copyright (C) Microsoft Corporation. All rights reserved.
[00:51:33][Step 2/10] 
[00:51:38][Step 2/10]   AutoGuru.Shared.Utilities -> /src/AutoGuru.Shared.Utilities/src/bin/Release/netstandard2.0/AutoGuru.Shared.Utilities.dll
[00:51:38][Step 2/10] /usr/share/dotnet/sdk/2.2.207/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(129,5): error MSB4018: The "GenerateDepsFile" task failed unexpectedly. [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
[00:51:38][Step 2/10] /usr/share/dotnet/sdk/2.2.207/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(129,5): error MSB4018: System.IO.IOException: The process cannot access the file '/src/AutoGuru.Shared.Utilities/src/bin/Release/netstandard2.0/AutoGuru.Shared.Utilities.deps.json' because it is being used by another process. [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
[00:51:38][Step 2/10] /usr/share/dotnet/sdk/2.2.207/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(129,5): error MSB4018:    at System.IO.FileStream.Init(FileMode mode, FileShare share) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
[00:51:38][Step 2/10] /usr/share/dotnet/sdk/2.2.207/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(129,5): error MSB4018:    at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
[00:51:38][Step 2/10] /usr/share/dotnet/sdk/2.2.207/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(129,5): error MSB4018:    at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
[00:51:38][Step 2/10] /usr/share/dotnet/sdk/2.2.207/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(129,5): error MSB4018:    at System.IO.File.Create(String path) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
[00:51:38][Step 2/10] /usr/share/dotnet/sdk/2.2.207/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(129,5): error MSB4018:    at Microsoft.NET.Build.Tasks.GenerateDepsFile.ExecuteCore() [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
[00:51:38][Step 2/10] /usr/share/dotnet/sdk/2.2.207/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(129,5): error MSB4018:    at Microsoft.NET.Build.Tasks.TaskBase.Execute() [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
[00:51:38][Step 2/10] /usr/share/dotnet/sdk/2.2.207/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(129,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
[00:51:38][Step 2/10] /usr/share/dotnet/sdk/2.2.207/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(129,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]

Second run worked. Makes you wonder if there’s something in the preceding step holding on to a file, like the dotnet restore I do just before.