NUnit3TestAdapter 3.15 not running tests with custom TestCaseSource

NUnit 3.12.0 NUnit3TestAdapter 3.15.0 Visual Studio 2019 Version 16.1.3 .NET Framework 4.6.1

namespace NUnitTest
{
   public class UnitTest1
    {
        public class DataProvider<T> where T : TestData, new()
        {
            public static IEnumerable<TestCaseData> GetData()
            {
                string[] names = new string[] { "Matthew", "Kelly", "Shawn" };
                char[] genders = new char[] { 'M', 'F', 'M' };
                int[] ages = new int[] { 35, 34, 30 };

                for (int i = 0; i < names.Length; i++)
                {
                    TestData testData = new TestData()
                    {
                        Name = names[i],
                        Gender = genders[i],
                        Age = ages[i]
                    };

                    TestCaseData tcd = new TestCaseData(testData);
                    tcd.SetName("{m} [" + testData.Name + " - " + testData.Gender + " - " + testData.Age + "]");

                    yield return tcd;
                }
            }
        }


        [Test]
        [TestCaseSource(typeof(DataProvider<TestData>), "GetData")]
        public void Persons(TestData test)
        {
            Assert.IsTrue(test.Name == "Matthew");
        }
    }
}

The above code correctly generates 3 test cases that have the following names in the Test Explorer:

Persons [Kelly - F - 34]
Persons [Matthew - M - 35]
Persons [Shawn - M - 30]

However, when I attempt to run/debug any of those test cases, they disappear from the Test Explorer, and nothing happens. A simple rebuild (or another clean and build) brings them back, but they still cannot be executed as they just disappear again.

Using the debugger, the custom data

If I downgrade to NUnit3TestAdapater 3.14, everything works as expected, I get 2 failed and 1 passed.

[OsirisTerje]

Thanks @MattKeenum ! Based on this case, in addition to the other, I am pretty sure we need to rollback 3.15. That performance work is unfortunately not finished.

[MattKeenum]

Edit: Just to be clear, I am only talking about how the Test Explorer displays the names of the tests. All of the other functionality works great.

@jnm2 Sure. Our company has over 500 applications that all need automated testing. Since we are mostly a C# shop, we chose NUnit as the testing framework of choice. As each team developed their own framework to write automated tests, we realized that every team was solving the same problems in different ways.

We wrote a generic wrapper around NUnit and distribute it as a NuGet package. This lets us solve the common problems once, and easily allows team members to move from team to team without having to learn a different framework.

Apart of this, we heavily customize the TestCaseSource to be able to create tests from different data sources, like Excel, CSV, Databases, etc. We then duplicate these tests changing slight configurations, like which environment to test, where the tests are executed (Local, Selenium Grid, Sauce Labs, etc.). If the team is using mobile devices, it allows the tests to be duplicated for different devices. This allows us to build once and run the tests we need to when approaching the CI/CD pipeline.

Unfortunately, I am prohibited by my company to upload pictures, but here are some example test names:

[Local : Dev : Win : Chrome] - [Card] - [MyMethod1] - [MyDataSourceTestName] [Grid : Dev : Win : Chrome] - [Card] - [MyMethod1] - [MyDataSourceTestName] [Sauce Labs : QA : Mac : Safari] - [Card] - [MyMethod2] - [MyDataSourceTestName]

Having full control over how the names are presented, makes it much easier to look through the Test Explorer for the tests the end user wants to run.

I’m open to ideas. We aren’t 100% sold on SetNames. Just realize that any changes we make to the way the Test Names are displayed in the Test Explorer, affects 100s of people using the framework. So we want to be consistent in how we display these names to reduce confusion among the teams.