Listener's `onAfterClass` is called before `@AfterClass` configuration methods are executed.

TestNG Version

7.6.1 and older

Expected behavior

I would expect that listener’s onAfterClass method is executed after onConfiguration* methods for all methods annotated with @AfterClass are called. Similarly to how onBeforeClass is called before the first method annotated with @BeforeClass is called.

Or at least that the ordering is consistent for both types of configuration methods.

Actual behavior

• onBeforeClass is called before configuration methods annotated w/ @BeforeClass are executed.
• onAfterClass is called before configuration methods annotated w/ @AfterClass are executed.

Is the issue reproducible on runner?

Issue is reproducible both on Gradle and Maven runners, did not check anything else. Here is a small Gradle project with listener and test case implementations that illustrates the problem: https://github.com/pshevche/testng-after-class-order-reproducer

Test case sample

• Test case: https://github.com/pshevche/testng-after-class-order-reproducer/blob/main/lib/src/test/java/dev/pshevche/AfterClassTest.java
• Listener implementation: https://github.com/pshevche/testng-after-class-order-reproducer/blob/main/lib/src/main/java/dev/pshevche/LoggingListener.java

Actual output:

onBeforeClass: dev.pshevche.AfterClassTest
onConfigurationSuccess: setupClass
onTestStart: test1
onTestSuccess: test1
onAfterClass: dev.pshevche.AfterClassTest
onConfigurationSuccess: cleanupClass

[juherr]

@leonard84 We already added some engine configurations for behavior changes in the past. The current one could use the same mechanism.

[leonard84]

@cbeust if you are really worried that a fix would break existing users, would you consider introducing a new listener interface, that has the “correct” behavior, or introduce a new default method onAfterClassAfterMethods that gets called at the correct time?