Improvement: ClassCleanup method to trigger after worker has executed all test methods not after all workers are complete

When running multiple [TestClass]es at the same time, why does the [ClassCleanup()] methods only get called at the very end?

File: UnitTest1.cs

    [TestClass]
    public class UnitTest1
    {
        [ClassInitialize()]
        public static void ClassInit(TestContext context) { }
    
        [ClassCleanup()]
        public static void ClassCleanup() { }
    
        [TestMethod]
        public void TestMethod() { }
    }

File: UnitTest2.cs

    [TestClass]
    public class UnitTest2
    {
        [ClassInitialize()]
        public static void ClassInit(TestContext context) { }
    
        [ClassCleanup()]
        public static void ClassCleanup() { }
    
        [TestMethod]
        public void TestMethod() { }
    }

The execution order is the following:

1. UnitTest2.ClassInit()
2. UnitTest2.TestMethod()
3. UnitTest1.ClassInit()
4. UnitTest1.TestMethod()
5. UnitTest2.ClassCleanup()
6. UnitTest1.ClassCleanup()

Notice that both [ClassCleanup] methods are executed at the end of the set; not after each TestClass is “done”.

But I expected a different behavior:

1. UnitTest2.ClassInit()
2. UnitTest2.TestMethod()
3. UnitTest2.ClassCleanup() - Expected
4. UnitTest1.ClassInit()
5. UnitTest1.TestMethod()
6. UnitTest1.ClassCleanup() - Expected

Microsoft’s Documentation - ClassCleanupAttribute Class says, “Identifies a method that contains code to be used after ALL the tests in the test class have run and to free resources obtained by the test class.”

But it appears to be run/executed late, after ALL tests have run. This seems wrong, and it is blocking my development.

[ScottMichaud]

Assembly and class cleanup today run at the end. Could you please elaborate on why this would block your development ?

It looks like the thread has already made some valid use cases, but here’s another example: Using Appium.WebDriver to run integration tests on a single-instance application with non-trivial load and close.

I would expect that each TestClass could be used to launch the application into a specific state, run the unit tests on that state, then close the application. Because the ClassCleanup doesn’t run before the next class attempts to start testing, the driver attempts to launch the application a second time in a different mode while the first instance is still open.

We’re currently using TestInitialize and TestCleanup to allow a “Run All” to succeed, but this significantly limits our ability to break apart our integration tests into separate, well defined TestMethods due to how much execution time it adds.

Of course, initialization and clean-up at arbitrary levels of granularity (ex: groups) could help minimize execution time further, by batching tests to minimize state changes within an application run, but TestClasses would cover most of that need (by cutting out the load and shutdown cycle).

[Haplois]

This is fixed in https://github.com/microsoft/testfx/pull/968 and released with 2.2.8. To enable it you can set ClassCleanupBehavior to ClassCleanupBehavior.EndOfClass in your ClassCleanup attribute.

This can also be set on assembly level by ClassCleanupExecution assembly level attribute.