Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Multiple unit test failures when running tests locally
See original GitHub issue[jw@cn06 arcade]$ ./test.sh
Running tests: /home/jw/dotnet/arcade/artifacts/bin/Microsoft.DotNet.NuGetRepack.Tests/Debug/netcoreapp3.1/Microsoft.DotNet.NuGetRepack.Tests.dll [netcoreapp3.1|x64]
Running tests: /home/jw/dotnet/arcade/artifacts/bin/Microsoft.DotNet.VersionTools.Tests/Debug/netcoreapp3.1/Microsoft.DotNet.VersionTools.Tests.dll [netcoreapp3.1|x64]
Running tests: /home/jw/dotnet/arcade/artifacts/bin/Microsoft.DotNet.Helix.JobSender.Tests/Debug/netcoreapp3.1/Microsoft.DotNet.Helix.JobSender.Tests.dll [netcoreapp3.1|x64]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.VersionTools.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.VersionTools/tests/Microsoft.DotNet.VersionTools.Tests.csproj]
Running tests: /home/jw/dotnet/arcade/artifacts/bin/Microsoft.DotNet.Arcade.Sdk.Tests/Debug/netcoreapp3.1/Microsoft.DotNet.Arcade.Sdk.Tests.dll [netcoreapp3.1|x64]
Running tests: /home/jw/dotnet/arcade/artifacts/bin/Microsoft.DotNet.Helix.Sdk.Tests/Debug/netcoreapp3.1/Microsoft.DotNet.Helix.Sdk.Tests.dll [netcoreapp3.1|x64]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.NuGetRepack.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.NuGetRepack/tests/Microsoft.DotNet.NuGetRepack.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.Helix.JobSender.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.Helix/JobSender.Tests/Microsoft.DotNet.Helix.JobSender.Tests.csproj]
Running tests: /home/jw/dotnet/arcade/artifacts/bin/Microsoft.DotNet.SourceBuild.Tasks.Tests/Debug/netcoreapp3.1/Microsoft.DotNet.SourceBuild.Tasks.Tests.dll [netcoreapp3.1|x64]
Running tests: /home/jw/dotnet/arcade/artifacts/bin/Microsoft.DotNet.RemoteExecutor.Tests/Debug/netcoreapp3.1/Microsoft.DotNet.RemoteExecutor.Tests.dll [netcoreapp3.1|x64]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.Arcade.Sdk.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.Arcade.Sdk.Tests/Microsoft.DotNet.Arcade.Sdk.Tests.csproj]
Running tests: /home/jw/dotnet/arcade/artifacts/bin/Microsoft.DotNet.PackageValidation.Tests/Debug/netcoreapp3.1/Microsoft.DotNet.PackageValidation.Tests.dll [netcoreapp3.1|x64]
Running tests: /home/jw/dotnet/arcade/artifacts/bin/Microsoft.DotNet.VersionTools.Tasks.Tests/Debug/netcoreapp3.1/Microsoft.DotNet.VersionTools.Tasks.Tests.dll [netcoreapp3.1|x64]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.Helix.Sdk.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.Helix/Sdk.Tests/Microsoft.DotNet.Helix.Sdk.Tests/Microsoft.DotNet.Helix.Sdk.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.RemoteExecutor.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.RemoteExecutor/tests/Microsoft.DotNet.RemoteExecutor.Tests.csproj]
Running tests: /home/jw/dotnet/arcade/artifacts/bin/Microsoft.DotNet.Build.Tasks.Feed.Tests/Debug/netcoreapp3.1/Microsoft.DotNet.Build.Tasks.Feed.Tests.dll [netcoreapp3.1|x64]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.PackageValidation.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.PackageValidation.Tests/Microsoft.DotNet.PackageValidation.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.VersionTools.Tasks.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.VersionTools.Tasks.Tests/Microsoft.DotNet.VersionTools.Tasks.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.Build.Tasks.Feed.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.Build.Tasks.Feed.Tests/Microsoft.DotNet.Build.Tasks.Feed.Tests.csproj]
Running tests: /home/jw/dotnet/arcade/artifacts/bin/Microsoft.DotNet.ApiCompat.Tests/Debug/netcoreapp3.1/Microsoft.DotNet.ApiCompat.Tests.dll [netcoreapp3.1|x64]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.ApiCompat.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.ApiCompat/tests/Microsoft.DotNet.ApiCompat.Tests.csproj]
Running tests: /home/jw/dotnet/arcade/artifacts/bin/Microsoft.DotNet.SignTool.Tests/Debug/netcoreapp3.1/Microsoft.DotNet.SignTool.Tests.dll [netcoreapp3.1|x64]
Running tests: /home/jw/dotnet/arcade/artifacts/bin/Microsoft.DotNet.XUnitExtensions.Tests/Debug/netcoreapp3.1/Microsoft.DotNet.XUnitExtensions.Tests.dll [netcoreapp3.1|x64]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.SignTool.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.SignTool.Tests/Microsoft.DotNet.SignTool.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.XUnitExtensions.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.XUnitExtensions/tests/Microsoft.DotNet.XUnitExtensions.Tests.csproj]
Running tests: /home/jw/dotnet/arcade/artifacts/bin/Microsoft.DotNet.Build.Tasks.Packaging.Tests/Debug/netcoreapp3.1/Microsoft.DotNet.Build.Tasks.Packaging.Tests.dll [netcoreapp3.1|x64]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.Build.Tasks.Packaging.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.Build.Tasks.Packaging/tests/Microsoft.DotNet.Build.Tasks.Packaging.Tests.csproj]
Tests succeeded: /home/jw/dotnet/arcade/artifacts/bin/Microsoft.DotNet.SourceBuild.Tasks.Tests/Debug/netcoreapp3.1/Microsoft.DotNet.SourceBuild.Tasks.Tests.dll [netcoreapp3.1|x64]
Build FAILED.
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.VersionTools.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.VersionTools/tests/Microsoft.DotNet.VersionTools.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.NuGetRepack.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.NuGetRepack/tests/Microsoft.DotNet.NuGetRepack.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.Helix.JobSender.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.Helix/JobSender.Tests/Microsoft.DotNet.Helix.JobSender.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.Arcade.Sdk.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.Arcade.Sdk.Tests/Microsoft.DotNet.Arcade.Sdk.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.Helix.Sdk.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.Helix/Sdk.Tests/Microsoft.DotNet.Helix.Sdk.Tests/Microsoft.DotNet.Helix.Sdk.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.RemoteExecutor.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.RemoteExecutor/tests/Microsoft.DotNet.RemoteExecutor.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.PackageValidation.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.PackageValidation.Tests/Microsoft.DotNet.PackageValidation.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.VersionTools.Tasks.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.VersionTools.Tasks.Tests/Microsoft.DotNet.VersionTools.Tasks.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.Build.Tasks.Feed.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.Build.Tasks.Feed.Tests/Microsoft.DotNet.Build.Tasks.Feed.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.ApiCompat.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.ApiCompat/tests/Microsoft.DotNet.ApiCompat.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.SignTool.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.SignTool.Tests/Microsoft.DotNet.SignTool.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.XUnitExtensions.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.XUnitExtensions/tests/Microsoft.DotNet.XUnitExtensions.Tests.csproj]
XUnit : error : Tests failed: /home/jw/dotnet/arcade/artifacts/log/Debug/Microsoft.DotNet.Build.Tasks.Packaging.Tests_netcoreapp3.1_x64.log [netcoreapp3.1|x64] [/home/jw/dotnet/arcade/src/Microsoft.DotNet.Build.Tasks.Packaging/tests/Microsoft.DotNet.Build.Tasks.Packaging.Tests.csproj]
0 Warning(s)
13 Error(s)
Time Elapsed 00:00:14.62
Build failed with exit code 1. Check errors above.
[jw@cn06 arcade]$
Issue Analytics
- State:
- Created 3 years ago
- Comments:14 (14 by maintainers)
Top Results From Across the Web
c# - Unit test fails when running all, but passes when ...
I have had this issue before and it boiled down to two tests running at the same time using some shared piece of...
Read more >Unit test fails in Azure DevOps but passes locally
When tests are run using Test Explorer "Run all" it works locally, but when tests are run as part of the build process...
Read more >Why your tests may pass locally but fail in Jenkins
Symptom: Tests pass locally when run in a certain order, but fail in Jenkins when run in a different order (or tests pass/fail...
Read more >How to Fix Flaky Tests - Semaphore CI
A test that intermittently fails for no apparent reason — or works in your local machine and fails with continuous integration — is...
Read more >Flaky tests resolving using Test Repetitions in Xcode
Flaky tests can be resolved using test repetitions and several best practices when writing tests and running tests locally.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
[Async Triage] I favor closing this as fixed, with perhaps just the last step being adding a “skip if in VS” attribute on the tests, and perhaps some comments on these remote executor tests that it E_FAIL’s when run from VS. The tests work from the console, so arcade users can run the tests locally before submitting PRs, and it’s a special component for IPC and other cross process scenarios for the runtime team, not really shared infra for everyone (various things in Arcade are like this)
It sounds like the only bits left are “RemoteExecutor”… But it’s not really something we own. Is FR a place where we clean up every test failure in code that people check into arcade? That’s… a pretty intense burden, and not particularly sustainable (people check in broken code to arcade, and we have to deal with it)
I’d probably just slap “ignored” on all these tests as part of FR.
Triage, thoughts?