question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

DataTestMethod skips DataRows that provide different data arrays, only keeping last

See original GitHub issue

Steps to reproduce

[DataTestMethod]
[DataRow(0, new int[] { })]
[DataRow(0, new int[] { 0 })]
[DataRow(0, new int[] { 0, 0, 0 })] // Is executed
[DataRow(1, new int[] { 1 })]
[DataRow(1, new int[] { 0, 1 })] // Is executed
public void TestMethod1(int expectedSum, int[] buckets)
{
    Assert.AreEqual(expectedSum, buckets.Sum());
}

Expected behavior

Test is executed with 5 sets of arguments.

Actual behavior

Test is executed with only 2 sets of arguments (see comments in snippet).

image

Test discovery DOES see all 5 test cases:

========== Starting test discovery ==========
========== Test discovery finished: 5 Tests found in 450,8 ms ==========
========== Starting test run ==========
========== Test run finished: 2 Tests (2 Passed, 0 Failed, 0 Skipped) run in 69 ms ==========

Environment

Windows 11, Visual Studio 2022 v17.0.1, .NET SDK 6.0.100.

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.11.0" />
    <PackageReference Include="MSTest.TestAdapter" Version="2.2.7" />
    <PackageReference Include="MSTest.TestFramework" Version="2.2.7" />
    <PackageReference Include="coverlet.collector" Version="3.1.0" />
  </ItemGroup>

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:3
  • Comments:6 (4 by maintainers)

github_iconTop GitHub Comments

2reactions
Hejlecommented, Jul 13, 2022

I have the same problem, with this very simple Unittest, which should fail at execution 2, but passes because of the bug making it only run the first test. However if I run the test from the Commandline using dotnet test, it correctly fails. 😃

[DataTestMethod]
[DataRow("")] //Test is executed
[DataRow(null)] //Test is ignored
public void TestNotNull(string input)
{
    //Assert
    Assert.IsNotNull(input);
}

For others who have the same problem, a simple workaround is setting the DisplayName in the DataRow:

[DataTestMethod]
[DataRow("", DisplayName = $"{nameof(TestNotNull)}_WithEmptyString")]
[DataRow(null, DisplayName = $"{nameof(TestNotNull)}_WithNullString")]
public void TestNotNull(string input)
{
    //Assert
    Assert.IsNotNull(input);
}
1reaction
Evangelinkcommented, Oct 11, 2022

We are working on the fix as I write these lines. I just wanted to mention/keep track of the fact that doing a test with Microsoft.NET.Test.Sdk version 17.3.2 and MSTest version 2.2.10 I can correctly reproduce the issue in VS (only 2 tests appear) but result is correct in command line (5 tests appear).

Read more comments on GitHub >

github_iconTop Results From Across the Web

visual studio - Only first DataRow gets executed in MSTest
If you're like me and were attempting to run a DataTestMethod on arrays, I think the issue is that visual studio will not...
Read more >
Saving Time with the DataTestMethod Attribute
With the DataTestMethod attribute, we can just put these data parameters in an attribute on the method itself. This method is built into...
Read more >
C# - Using the DynamicData attribute in unit tests
With DataRow, the problem is you can only pass in constants and arrays. You can't pass in reference types. When you try to...
Read more >
Create Data-Driven Unit Tests - Visual Studio (Windows)
Learn how to use the Microsoft unit test framework for managed code to set up a unit test method to retrieve values from...
Read more >
MSTest v2: Data tests
MSTest v2 provides 3 ways to create parametrized tests. #Using DataRow. The [DataRow] attribute allows setting the values of the parameter of ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found