dotnet build fails on .sln with IOException
See original GitHub issueRecently, and all of a sudden, we’re consistently running into a race condition in our builds. Initially I believed this was caused by the fact we build passing the .sln file and were also passing a runtime identifier (rid), but even without that the problem still occurs quite regularly.
I’ve deleted my comments off https://github.com/dotnet/sdk/issues/863 as it’s no longer an issue with just specifying the rid, something else is going on. I’ve re-iterated them in here.
Scenario A docker container build. I’m specifically building a base image used by all our individual microservice images to speed things up. As such, restoring and building by pointing at the .sln file makes most sense. (It’s not an exact science, but… that’s Docker builds…)
My builder Dockerfile is doing:
# Restore NuGet packages
RUN dotnet restore Microservices.sln # not doing this anymore: -r linux-musl-x64
# Copy across the rest of the source files
WORKDIR ..
COPY ./ ./
# Build entire sln
RUN dotnet build Microservices/Microservices.sln -c Release --no-restore # not doing this anymore: -r linux-musl-x64
My services Dockerfile is doing this:
# Test
RUN dotnet test ../Tests/${SERVICE_NAME}.Tests/${SERVICE_NAME}.Tests.csproj -c Release --no-restore
# Publish
RUN dotnet publish -c Release --no-restore -o /dist # not doing this anymore: -r linux-musl-x64
My docker-compose.yml is essentially specifying that each service builds itself based on the common microservices-builder image:
version: '3.7'
services:
microservices-builder:
image: microservices-builder
build:
context: ../..
dockerfile: Microservices/Docker/builder/Dockerfile
booking.service:
image: booking.service
depends_on:
- "microservices-builder"
build:
context: ..
dockerfile: BookingService/Dockerfile
ports:
- "33000:80"
rating.service:
... etc
Expected Build succeeds.
Actual
Build fails due to a IOException (used by another process) on a shared library deps.json file.
The specific step is the RUN dotnet build ...
in the builder Dockerfile.
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018: The "GenerateDepsFile" task failed unexpectedly. [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018: System.IO.IOException: The process cannot access the file '/src/AutoGuru.Shared.Utilities/src/bin/Release/netstandard2.0/AutoGuru.Shared.Utilities.deps.json' because it is being used by another process. [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018: at System.IO.FileStream.Init(FileMode mode, FileShare share, String originalPath) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018: at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018: at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018: at System.IO.File.Create(String path) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018: at Microsoft.NET.Build.Tasks.GenerateDepsFile.WriteDepsFile(String depsFilePath) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018: at Microsoft.NET.Build.Tasks.GenerateDepsFile.ExecuteCore() [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018: at Microsoft.NET.Build.Tasks.TaskBase.Execute() [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
10>/usr/share/dotnet/sdk/3.1.201/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(194,5): error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [/src/AutoGuru.Shared.Utilities/src/AutoGuru.Shared.Utilities.csproj]
What I’ve tried
- No longer specify the rid in the dotnet cli commands, instead specify it in the Directory.Build.props file conditionally when I know a docker build is happening. (initially was doing this only on project files which ended in Service.csproj, but removed that and still the same)
<PropertyGroup>
<RuntimeIdentifier Condition="'$(AUTOGURU_IS_DOCKER_BUILD)' == 'True'">linux-musl-x64</RuntimeIdentifier>
</PropertyGroup>
- Uncheck “Build” for the .dcproj in Release configuration, just so that’s not doing anything as it doesn’t need to.
- Attempt to disable a parallel build by passing
--disable-parallel
todotnet build
.
Even though the Dockerfile for my services is doing a specific dotnet test
and a dotnet publish
in the specific project’s directory, the intent is really that these don’t need to do much (–no-restore is passed, etc.) as the image it’s based on from the first Dockerfile has done all the hard work and cached everything along the way. This is essential in docker builds to speed up things.
Any help would be greatly appreciated.
Issue Analytics
- State:
- Created 3 years ago
- Comments:10 (1 by maintainers)
I had the same issue on ubuntu. I have installed donet sdk 3.1.401.
I tried a lot of things like doing a restore separately and calling the
build-server shutdown
command. Nothing worked.Then I tried the suggestion of @rainersigwald and set
-maxcpucount=1
flag and ran it. It worked.I looked at my project structure and there are 7 projects in the solution all referencing other projects in a tree like manner. But there were additional references like this.
I updated the references in such a way that trainsient references are not stated in the csproj file.
And this fixed the issue.
Though I would have thought that the build command would handle that.
Also no idea what would happen in the following scenario
Here project D is references by 2 independant projects. This will most likely give the same error as above.
FYI, @wli3’s suggestion to shutdown the build server appears to have not worked. Just saw this on TeamCity (on our current dotnetcore 2.2 branch). Again the culprit is AutoGuru.Shared.Utilities.csproj. Interesting. I’ll definitely try the cpu max count switch if it keeps causing me issues.
Second run worked. Makes you wonder if there’s something in the preceding step holding on to a file, like the
dotnet restore
I do just before.