Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

NUnit3TestAdapter 3.15.0 fails to run test: "NUnit failed to load"

See original GitHub issue

When reporting a bug, please provide the following information to speed up triage:

  • NUnit and NUnit3TestAdapter versions

Matches App.sln in repo below:

<PackageReference Include="nunit" Version="3.8.1" />
<PackageReference Include="NUnit3TestAdapter" Version="3.15.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.2.0" />
  • Visual Studio edition and full version number (see Help About)

16.2.3

  • A short repro, preferably attached or pointing to a git repo or gist

Full reproduction https://github.com/seangwright/NUnit-Test-Adapter-Bug-Repro

App.sln is using

<PackageReference Include="nunit" Version="3.8.1" />
<PackageReference Include="NUnit3TestAdapter" Version="3.15.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.2.0" />

App2.sln is using

<PackageReference Include="nunit" Version="3.8.1" />
<PackageReference Include="NUnit3TestAdapter" Version="3.14.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.2.0" />

Test fails to run for NUnit3TestAdapter 3.15.0 but runs correctly for 3.14.0

  • What .net platform and version is being targeted

net472 (but with the new Common Project System / SDK style project)

  • If TFS/VSTS issue, what version, hosted, on-premises, and what build task you see this in

On-prem in Visual Studio 2019

Output Window - Tests (for 3.14.0)

[8/26/2019 6:02:11.191 PM Diagnostic] Enqueue operation 'RunSelectedOperation', hashcode:16874580 
[8/26/2019 6:02:11.192 PM Diagnostic] Operation left in the the queue: 1
[8/26/2019 6:02:11.192 PM Diagnostic] 	'RunSelectedOperation', hashcode:16874580
[8/26/2019 6:02:11.192 PM Diagnostic] 

[8/26/2019 6:02:11.192 PM Diagnostic] Operation Dequeue : 'RunSelectedOperation'
[8/26/2019 6:02:11.548 PM Diagnostic] Starting programmatic build of containers... 
[8/26/2019 6:02:11.605 PM Diagnostic] Loading Project 4b5691bb-da6f-4216-89bf-09783092ca8a [1] => 
===
Id: 4b5691bb-da6f-4216-89bf-09783092ca8a
ProjectFilePath: C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App2.Tests\App2.Tests.csproj
DefaultOutputPath: C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App2.Tests\bin\Debug\net472\App2.Tests.dll
DefaultTargetFramework: net472
ProjectName: App2.Tests
Capabilities: UseFileGlobs,AppDesigner,AssemblyReferences,Managed,SupportAvailableItemName,FolderPublish,CPS,ProjectConfigurationsDeclaredDimensions,RelativePathDerivedDefaultNamespace,OpenProjectFile,UserSourceItems,VisualStudioWellKnownOutputGroups,TestContainer,DynamicDependentFile,NoGeneralDependentFileIcon,CSharp,SharedProjectReferences,Publish,ReferenceManagerProjects,EditAndContinue,Microsoft.VisualStudio.ProjectSystem.RetailRuntime,ReferenceManagerWinRT,AllTargetOutputGroups,SingleFileGenerators,ProjectReferences,ReferenceManagerSharedProjects,HostSetActiveProjectConfiguration,ClassDesigner,PackageReferences,GenerateDocumentationFile,AppSettings,HandlesOwnReload,RunningInVisualStudio,WinRTReferences,Pack,LaunchProfiles,ReferenceManagerCOM,LanguageService,ReferenceManagerAssemblies,ReferenceManagerBrowse,PersistDesignTimeDataOutOfProject,DependenciesTree,DeclaredSourceItems,PreserveFormatting,.NET,DataSourceWindow,OutputGroups,COMReferences
IsAppContainer: False
IsCpsProject: True
IsMiscellaneous: False
HasTestContainerCapability: True
IsCSharpOrVBProject: True

[8/26/2019 6:02:12.417 PM Diagnostic] Completed programmatic build of containers.
[8/26/2019 6:02:12.422 PM Diagnostic] TestContainer update (build) complete : 878 ms
[8/26/2019 6:02:12.524 PM Diagnostic] DiscoveryOperation<RunSelectedOperation> FinishedChangedCotainers, changed container count is 1
[8/26/2019 6:02:12.557 PM Diagnostic] Discovering the following containers :
[8/26/2019 6:02:12.557 PM Diagnostic] 	C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App2.Tests\bin\Debug\net472\App2.Tests.dll
[8/26/2019 6:02:12.608 PM Informational] ---------- Discovery started ----------
[8/26/2019 6:02:12.608 PM Diagnostic] TelemetrySession: Creating the event: VS/UnitTest/TestWindow/Ext/RunSettingsService 
[8/26/2019 6:02:12.608 PM Diagnostic] Event:VS/UnitTest/TestWindow/Ext/RunSettingsService key: VS.UnitTest.TestWindow.RunSettingsService.Name value:VSTest Run Configuration
[8/26/2019 6:02:12.608 PM Diagnostic] TelemetrySession: Creating the event: VS/UnitTest/TestWindow/Ext/RunSettings 
[8/26/2019 6:02:12.608 PM Diagnostic] Event:VS/UnitTest/TestWindow/Ext/RunSettings key: VS.UnitTest.TestWindow.RunSettings.Services value:1
[8/26/2019 6:02:12.609 PM Diagnostic] File timestamp changed from 8/26/2019 5:52:15 PM to 8/26/2019 6:02:12 PM for C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App2.Tests\bin\Debug\net472\App2.Tests.dll
[8/26/2019 6:02:12.610 PM Diagnostic] Grouped C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App2.Tests\bin\Debug\net472\App2.Tests.dll : (AnyCPU, Framework45, net472, )
[8/26/2019 6:02:12.611 PM Diagnostic] Test discovery settings for C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App2.Tests\bin\Debug\net472\App2.Tests.dll:
 <RunSettings>
  <RunConfiguration>
    <ResultsDirectory>C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\TestResults</ResultsDirectory>
    <SolutionDirectory>C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\</SolutionDirectory>
    <TargetPlatform>X86</TargetPlatform>
    <CollectSourceInformation>False</CollectSourceInformation>
  </RunConfiguration>
</RunSettings>.
[8/26/2019 6:02:13.309 PM Informational] NUnit Adapter 3.13.0.0: Test discovery starting
[8/26/2019 6:02:13.602 PM Informational] NUnit Adapter 3.13.0.0: Test discovery complete
[8/26/2019 6:02:13.697 PM Informational] ========== Discovery finished: 1 tests found (0:00:01.0457653) ==========
[8/26/2019 6:02:13.764 PM Informational] ---------- Run started ----------
[8/26/2019 6:02:13.790 PM Diagnostic] TelemetrySession: Creating the event: VS/UnitTest/TestWindow/Ext/RunSettingsService 
[8/26/2019 6:02:13.790 PM Diagnostic] Event:VS/UnitTest/TestWindow/Ext/RunSettingsService key: VS.UnitTest.TestWindow.RunSettingsService.Name value:VSTest Run Configuration
[8/26/2019 6:02:13.790 PM Diagnostic] TelemetrySession: Creating the event: VS/UnitTest/TestWindow/Ext/RunSettings 
[8/26/2019 6:02:13.790 PM Diagnostic] Event:VS/UnitTest/TestWindow/Ext/RunSettings key: VS.UnitTest.TestWindow.RunSettings.Services value:1
[8/26/2019 6:02:13.837 PM Diagnostic] Grouped C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App2.Tests\bin\Debug\net472\App2.Tests.dll : (AnyCPU, Framework45, net472, )
[8/26/2019 6:02:13.868 PM Diagnostic] Tests run settings for C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App2.Tests\bin\Debug\net472\App2.Tests.dll:
 <RunSettings>
  <RunConfiguration>
    <ResultsDirectory>C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\TestResults</ResultsDirectory>
    <SolutionDirectory>C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\</SolutionDirectory>
    <TargetPlatform>X86</TargetPlatform>
    <CollectSourceInformation>False</CollectSourceInformation>
  </RunConfiguration>
</RunSettings>.
[8/26/2019 6:02:14.167 PM Diagnostic] UpdateSummary Detail Unchanged: SKIPPED
[8/26/2019 6:02:14.456 PM Informational] NUnit Adapter 3.13.0.0: Test execution started
[8/26/2019 6:02:14.482 PM Informational] Running selected tests in C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App2.Tests\bin\Debug\net472\App2.Tests.dll
[8/26/2019 6:02:14.580 PM Diagnostic] UpdateSummary Detail Unchanged: SKIPPED
[8/26/2019 6:02:14.749 PM Informational]    NUnit3TestExecutor converted 1 of 1 NUnit test cases
[8/26/2019 6:02:14.897 PM Informational] NUnit Adapter 3.13.0.0: Test execution complete
[8/26/2019 6:02:15.264 PM Diagnostic] Test assembly C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App2.Tests\bin\Debug\net472\App2.Tests.dll references test framework: Nunit
[8/26/2019 6:02:15.273 PM Diagnostic] Project App2.Tests references test adapter: NUnit3TestAdapter, version 3.14.0
[8/26/2019 6:02:15.277 PM Informational] ========== Run finished: 1 tests run (0:00:01.3001368) ==========
[8/26/2019 6:02:15.355 PM Diagnostic] UpdateSummary Detail Unchanged: SKIPPED

Output Window - Tests (for 3.15.0)

[8/26/2019 5:57:01.653 PM Diagnostic] Enqueue operation 'RunSelectedOperation', hashcode:32775692 
[8/26/2019 5:57:01.653 PM Diagnostic] Operation left in the the queue: 1
[8/26/2019 5:57:01.653 PM Diagnostic] 	'RunSelectedOperation', hashcode:32775692
[8/26/2019 5:57:01.653 PM Diagnostic] 

[8/26/2019 5:57:01.653 PM Diagnostic] Operation Dequeue : 'RunSelectedOperation'
[8/26/2019 5:57:01.995 PM Diagnostic] Starting programmatic build of containers... 
[8/26/2019 5:57:02.243 PM Diagnostic] Completed programmatic build of containers.
[8/26/2019 5:57:02.243 PM Diagnostic] TestContainer update (build) complete : 248 ms
[8/26/2019 5:57:02.244 PM Diagnostic] DiscoveryOperation<RunSelectedOperation> FinishedChangedCotainers, changed container count is 1
[8/26/2019 5:57:02.244 PM Diagnostic] Discovering the following containers :
[8/26/2019 5:57:02.244 PM Diagnostic] 	C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App.Tests\bin\Debug\net472\App.Tests.dll
[8/26/2019 5:57:02.249 PM Informational] ---------- Discovery started ----------
[8/26/2019 5:57:02.263 PM Diagnostic] TelemetrySession: Creating the event: VS/UnitTest/TestWindow/Ext/RunSettingsService 
[8/26/2019 5:57:02.263 PM Diagnostic] Event:VS/UnitTest/TestWindow/Ext/RunSettingsService key: VS.UnitTest.TestWindow.RunSettingsService.Name value:VSTest Run Configuration
[8/26/2019 5:57:02.264 PM Diagnostic] TelemetrySession: Creating the event: VS/UnitTest/TestWindow/Ext/RunSettings 
[8/26/2019 5:57:02.264 PM Diagnostic] Event:VS/UnitTest/TestWindow/Ext/RunSettings key: VS.UnitTest.TestWindow.RunSettings.Services value:1
[8/26/2019 5:57:02.264 PM Diagnostic] File timestamp remains 8/26/2019 5:49:36 PM for C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App.Tests\bin\Debug\net472\App.Tests.dll
[8/26/2019 5:57:02.264 PM Informational] ========== Discovery skipped: All test containers are up to date ==========
[8/26/2019 5:57:02.276 PM Informational] ---------- Run started ----------
[8/26/2019 5:57:02.297 PM Diagnostic] TelemetrySession: Creating the event: VS/UnitTest/TestWindow/Ext/RunSettingsService 
[8/26/2019 5:57:02.298 PM Diagnostic] Event:VS/UnitTest/TestWindow/Ext/RunSettingsService key: VS.UnitTest.TestWindow.RunSettingsService.Name value:VSTest Run Configuration
[8/26/2019 5:57:02.298 PM Diagnostic] TelemetrySession: Creating the event: VS/UnitTest/TestWindow/Ext/RunSettings 
[8/26/2019 5:57:02.298 PM Diagnostic] Event:VS/UnitTest/TestWindow/Ext/RunSettings key: VS.UnitTest.TestWindow.RunSettings.Services value:1
[8/26/2019 5:57:02.299 PM Diagnostic] Grouped C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App.Tests\bin\Debug\net472\App.Tests.dll : (AnyCPU, Framework45, net472, )
[8/26/2019 5:57:02.299 PM Diagnostic] Tests run settings for C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App.Tests\bin\Debug\net472\App.Tests.dll:
 <RunSettings>
  <RunConfiguration>
    <ResultsDirectory>C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\TestResults</ResultsDirectory>
    <SolutionDirectory>C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\</SolutionDirectory>
    <TargetPlatform>X86</TargetPlatform>
    <CollectSourceInformation>False</CollectSourceInformation>
  </RunConfiguration>
</RunSettings>.
[8/26/2019 5:57:02.601 PM Diagnostic] UpdateSummary Detail Unchanged: SKIPPED
[8/26/2019 5:57:02.886 PM Informational] NUnit Adapter 3.15.0.0: Test execution started
[8/26/2019 5:57:02.914 PM Informational] Running selected tests in C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App.Tests\bin\Debug\net472\App.Tests.dll
[8/26/2019 5:57:02.996 PM Diagnostic] UpdateSummary Detail Unchanged: SKIPPED
[8/26/2019 5:57:03.162 PM Informational]    NUnit failed to load C:\dev\workspace\NUnit-Test-Adapter-Bug-Repro\App.Tests\bin\Debug\net472\App.Tests.dll
[8/26/2019 5:57:03.170 PM Informational] NUnit Adapter 3.15.0.0: Test execution complete
[8/26/2019 5:57:03.284 PM Diagnostic] Project App.Tests references test adapter: NUnit3TestAdapter, version 3.15.0
[8/26/2019 5:57:03.286 PM Informational] ========== Run finished: 0 tests run (0:00:00.9445475) ==========
[8/26/2019 5:57:03.526 PM Diagnostic] UpdateSummary Detail Unchanged: SKIPPED
[8/26/2019 5:57:03.901 PM Diagnostic] UpdateSummary Detail Unchanged: SKIPPED

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:12
  • Comments:68 (50 by maintainers)

github_iconTop GitHub Comments

1reaction
akarnokdcommented, Nov 15, 2019

FWIW, I figured out why the tests kept failing due to the timeout.

What happens is that when running the code on .NETCore3, The ThreadPool may end up filled with tasks blocked or doing NUnit work and the pool waits 500ms before creating a new worker thread. Some tests use code that execute Task.Run which would then wait multiples of 500 milliseconds to get running so they randomly timeout on my 4 core machine (no HT).

1reaction
OsirisTerjecommented, Nov 15, 2019

Yes, that is so. But they need to publish it, I just wonder if the maintainers are still around.
@jnm2 just told me the breaking change is at version 3.9. See comment higher up: https://github.com/nunit/nunit3-vs-adapter/issues/648#issuecomment-526252599

Read more comments on GitHub >

github_iconTop Results From Across the Web

nunit tests discovered but not running
Hi, I have NUnit 3.12 and Nunit3TestAdaptor 3.15.1 installed, using VS2019 community. I am able to discover the tests in the test explorer ......
Read more >
visual studio 19/22 Nunit test doesn't run by nunit adapter
You must install the NUnit 3 Test Adapter in Visual Studio 2019 NUnit 3 Test Adapter Extension. Go to Extensions -> Manage Extensions ......
Read more >
Adapter V3 Release Notes
648 NUnit3TestAdapter 3.15.0 fails to run test: "NUnit failed to load" (when using NUnit framework less than version 3.11).
Read more >
NUnit3TestAdapter 4.5.0
The NUnit 3 Test Adapter runs NUnit 3.x tests in Visual Studio 2012 and newer. You can download the latest release version alternate...
Read more >
C# – NUnit failed to load DLL
I am getting the following error message when trying to run unit tests in Visual Studio: NUnit failed to load w:\Repos\trading.tools\Trading.Tools.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

NUnit3TestAdapter 3.15 OneTimeSetUp not working anymore

Next page

Test Explorer shows excess class name for tests with multiple TestCaseAttribute