Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

SkipNonTestAssemblies does not skip non-test assemblies if they reference nunit.framework

See original GitHub issue

Summary

When a assembly references nunit.framework (e.g. containing a helper method that calls Assert), but does not contain any actual tests, nunit3-console exists with an error even though the --skipnontestassemblies parameter is specified.

Steps to reproduce:

Example project available at https://github.com/ap0llo/repro-nunit-console-skipnontestassemblies
Repo contains two assemblies, neither of which have any tests in them
- FailingAssembly contains a method calling Assert.True an thus has a reference to nunit.framework.dll
- WorkingAssembly has no reference to nunit.framework
Run both assemblies through nunit3-console:

nunit3-console.exe --skipnontestassemblies .\bin\net461\FailingAssembly.dll
nunit3-console.exe --skipnontestassemblies .\bin\net461\WorkingAssembly.dll

Expected behavior

No tests are executed, but both calls complete without an error

Actual behavior

Execution for FailingAssembly completes with error No test fixtures were found

NUnit Console Runner 3.6.1
Copyright (C) 2017 Charlie Poole

Runtime Environment
   OS Version: Microsoft Windows NT 10.0.17763.0
  CLR Version: 4.0.30319.42000

Test Files
    .\bin\net461\FailingAssembly.dll


Errors, Failures and Warnings

1) Invalid : bin\net461\FailingAssembly.dll
No test fixtures were found.

Run Settings
    DisposeRunners: True
    SkipNonTestAssemblies: True
    WorkDirectory: C:\Users\Andreas\Desktop\nunittest
    ImageRuntimeVersion: 4.0.30319
    ImageTargetFrameworkName: .NETFramework,Version=v4.6.1
    ImageRequiresX86: False
    ImageRequiresDefaultAppDomainAssemblyResolver: False
    NumberOfTestWorkers: 4

Test Run Summary
  Overall result: Failed
  Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
  Start time: 2019-05-17 20:01:30Z
    End time: 2019-05-17 20:01:31Z
    Duration: 0.936 seconds

Results (nunit3) saved as TestResult.xml

Is this a bug or is this the intended behavior?

Issue Analytics

State:
Created 4 years ago
Comments:5 (4 by maintainers)

Top GitHub Comments

1reaction

CharliePoolecommented, May 17, 2019

This is intended behavior.

The definition of a test assembly is one that references the nunit framework unless it is specifically marked as a nontest assembly using the NonTestAssemblyAttribute at the assembly level.

0reactions

CharliePoolecommented, May 17, 2019

I deliberately didn’t document it thinking only third party runner authors would need it. But here’s a more common use case.

Top Results From Across the Web

NonTestAssembly

This attribute can be used in connection with the command line option --skipnontestassemblies of the console to skip assemblies without failing.

Console Command Line

Skip any non-test assemblies specified - or assemblies containing NUnit.Framework.NonTestAssemblyAttribute, without error. --debug, Causes NUnit to break into ...

Nunit SkipNonTestAssemblies setting is throwing error on ...

I was reading the documentation and though that the setting called SkipNonTestAssemblies might help me with the issue where test discovery ...

API - NUnit3Settings.SkipNonTestAssemblies Property

Gets or sets a value indicating whether execution of the test run should skip any non-test assemblies specified, without error. Namespace: Cake.Common.

Running NUnit tests and examining results from within an ...

My meta-test project has references to each of the projects containing my exercises ... TestBuilder class in the nunit.framework.tests assembly, as well as...