Expected behavior of annotated methods on an @Test(groups="mygroup") annotated class

I have experienced a strange issue in TestNG 7+ that used to work in TestNG 6. Basically I have a suite defined that runs a group of mygroup and I have a few test classes annotated with @Test(groups="mygroup"). These classes also have a defined @BeforeClass and @BeforeMethod and @AfterMethod which are public void.

When I run my suite, all of the public functions inside the class are picked up and run as if they had an @Test annotation. That includes the public functions that have the above three listed annotations, which all fail with NPEs.

So, my question is, what is the expected behavior in this scenario and what should my annotations look like? Here’s a very rudimentary code sample of what I have for one of the tests:

package com.company.app.integration.suite.app;

@Test(groups = "mygroup")
public class AppRestTest {
    @BeforeClass
    public void setupRestTest() {
        // A bunch of setup that is run as a Test
    }

    @BeforeMethod(alwaysRun = true)
    public void beforeMethod(Method method) {
        System.out.println("Begin " + method.getName() + "..."); // NPE
    }

    @AfterMethod(alwaysRun = true)
    public void afterMethod(ITestResult result) {
        System.out.println("End " + result.getMethod().getMethodName()); // NPE
    }

    @Test
    public void testRestPostGetAll() {
        // works fine and passes as expected
    }

    @Test(groups="othergroup")
    public void testRestPostGetAllNoFilter() {
        // works fine but runs with both group names
    }
    
}

TestNG Version

7.0.0, 7.1.0, 7.3.0

Expected behavior

The previous behavior was that the annotated methods would be skipped and any annotated method with a group that shouldn’t be run was excluded. (It’s worth noting that I am working on an upgrade from TestNG 6.8.8 to 7.x.x and so if the expected behavior has changed that’s fine, I’m looking for clarification as this is not expressed anywhere in the docs I could find).

Actual behavior

Any public function, even TestNG annotated, in the Test class is run as a test when the Class itself has an included groups tag.

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

Test case sample

Test case provided above.

[krmahadevan]

@cardonator - I am going ahead and closing off this issue since it turns out to be resolved.

[cardonator]

Ahh, thanks for continuing to look into this. So I tried again on 7.3.0 after basically clearing out my maven cache on the demo project and that does indeed appear to fix the issue! When I was testing this before I was having issues with 7.3.0 testing due to conflicts with the IDE. It’s possible that the times I was “testing” with 7.3.0 I was actually somehow still using compiled classes with an earlier version of TestNG included.

I really appreciate the help researching and testing. Sorry it turned out to be already resolved!