Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

TestCaseSource causes incorrectly surviving mutations

See original GitHub issue

Describe the bug Using TestCaseSourceAttribute in NUnit for tests cases causes mutants not to be correctly killed.

Minimal example attached (N.B. this applies to any mutant and not just strings) TestCaseSourceExample.zip

Logs log-20210223.txt VsTest-log.txt

Expected behavior Expected the mutant to be killed as we have a test that checks that the Name property is equal to the expected value

Desktop (please complete the following information):

OS: Windows 10 and Ubuntu 20.04 Docker Container
Type of project: core
Framework Version: core 3.1
Stryker Version: beta 0.21.1

Additional context We can work around this by writing explicit tests per test case but it is not ideal due to duplication and we have a lot of tests that use this method.

Issue Analytics

State:
Created 3 years ago
Comments:6 (5 by maintainers)

Top GitHub Comments

3reactions

dupdobcommented, Feb 23, 2021

Thanks for you report and kudos for providing such specific reproduction conditions. I will look into this, but I am not confident about finding a (satisfactory) fix. We have a similar issues with xUnit (see #843), which is linked to the VSTest xUnit runner implementation. I will report findings when I have time to replicate

0reactions

dupdobcommented, Feb 24, 2021

Thanks for the feedback. Indeed, Stryker has little control over the test execution workflow, we are limited by what we can do with VsTest. I am looking into how to provide a better fix for this, but this would require significant design changes (assuming it can be done in the first place), so there are undergoing discussions about the best strategy for this; so not sure when it will be available (i.e. probably not in the next version).

Top Results From Across the Web

Simple mutation unit testing erroneously report survival of ...

Is caused by the test not failing when a mutation is done. ... maybe some future Stryker version can still mark this as...

Command Line Options | dextool

The typical usage of this option is to reset the mutants with alive status to unknown in order to conduct mutation testing again...

A partial oracle for uniformity statistics

This paper investigates the problem of testing implementations of ... terms of all descriptive statistics than at least one survived mutant.

Untitled

A good cause and effect topic, Jobmarketonline facebook, High risk loans for ... Winjas grosshandel bergsgatan, Nunit testcasesource excel, Deception bay ...

q649e

Rachel wenning attorney, Minecraft citycraft survival, Mesoscopic model, ... Two-finger test, Ll change password, Mutations in intronic sequences, ...