@Test methods from super class are re-executed for every sub class

TestNG Version 6.14.3

Expected behavior

The @Test methods from the super class are executed once no matter how many sub classes inherit the super class.

Actual behavior

The @Test methods are executed with each sub class.

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

Test case sample

public class BaseTest  extends SetupClass{
    @BeforeClass...
    @BeforeMethod...
    @AfterClass...
    @AfterMethod....

    @Test
    public void baseMethod(){
        System.out.println("base method");
    }
}

public class Test1 extends BaseTest{
    @Test
    public void test1(){
        System.out.println("test1");
    }
}

public class Test2 extends BaseTest{
    @Test
    public void test2(){
        System.out.println("test2");
    }
}

I am running these tests in parallel from a TestNG xml file that looks like this:

<suite name="Regression Suite" verbose="1" parallel="classes" thread-count="4">
    <parameter name="project" value="TestPrj"/>
    <parameter name="server" value="QA01"/>
    <test name="QA01 TestPrj">
        <parameter name="environment" value="macos"/>
        <groups>
            <run>
                <include name="regression"/>
            </run>
        </groups>
        <packages>
            <package name="test"/>
        </packages>
    </test>
</suite>

P.S. Editing code in github issues is incredibly bad…please move to something else.

[krmahadevan]

@vlad230

If you ask me, it wouldn’t make any sense to execute the tests in the super class multiple times.

What exactly are you trying to achieve? It would be great if you could please help elaborate or explain your scenario.

If this is the expected behaviour of TestNG right now, maybe we could add this as in improvement.

Am not sure that would be taken up as an enhancement because its counter intuitive . If you basically want the base class method to be executed exactly once, no matter how many child classes are present, then you could do it today in your test project by doing the following

1. Have your base class implement org.testng.IHookable
2. Within its run() method, add an edit check that would check if the base class method is already executed and run it if not executed.

Here’s a sample

Marker interface that indicates that a base class method should run only once

import static java.lang.annotation.ElementType.METHOD;
import static java.lang.annotation.ElementType.TYPE;

import java.lang.annotation.Retention;
import java.lang.annotation.Target;

@Retention(java.lang.annotation.RetentionPolicy.RUNTIME)
@Target({METHOD, TYPE})
public @interface RunOnce {
}

Here’s the state tracker singleton

import java.lang.reflect.Method;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

public class StateTracker {

  private static final StateTracker instance = new StateTracker();

  private final Map<String, Object> runOnceData = new ConcurrentHashMap<>();

  private StateTracker() {}

  public static StateTracker getInstance() {
    return instance;
  }

  boolean canExecute(Method method) {
    RunOnce runOnce = method.getAnnotation(RunOnce.class);
    String methodName = method.getName();
    boolean execute = true;
    if (runOnce != null) {
      if (runOnceData.containsKey(methodName)) {
        execute = false;
      } else {
        runOnceData.put(methodName, new Object());
      }
    }
    return execute;
  }
}

Here’s how the base class would look like

import org.testng.IHookCallBack;
import org.testng.IHookable;
import org.testng.ITestResult;
import org.testng.annotations.Test;

public class BaseTest implements IHookable {

  @Test
  @RunOnce
  public void baseclassTestMethod() {
    System.err.println("Base class test method executed");
  }

  @Override
  public void run(IHookCallBack callBack, ITestResult testResult) {
    boolean execute =
        StateTracker.getInstance()
            .canExecute(testResult.getMethod().getConstructorOrMethod().getMethod());
    if (execute) {
      callBack.runTestMethod(testResult);
    } else {
      testResult.setStatus(ITestResult.SKIP);
      testResult.setThrowable(new RuntimeException("Intentionally skipping"));
    }
  }
}

Here’s how the child classes would look like

import org.testng.annotations.Test;

public class ChildClass1 extends BaseTest {

  @Test
  public void childTest() {
    System.err.println(getClass().getName() + ".childTest() executed");
  }
}

import org.testng.annotations.Test;

public class ChildClass2 extends BaseTest {

  @Test
  public void childTest() {
    System.err.println(getClass().getName() + ".childTest() executed");
  }
}

You would need to add some filtering mechanism that prunes the test results and removes skipped tests which are annotated using @RunOnce

[krmahadevan]

@vlad230

thanks a lot for the detailed solution but it looks a bit cumbersome to me

Your use case also kind of looks a bit different from the regular. Moreover I have given you a working example of how it can be done. All you need to do is, just adopt this and build upon this.

Since TestNG can be customized, I would rather see this as the default behaviour.

No. That cannot be done in my opinion. Tests are meant to be executed all the time. Just because they reside in a base class doesn’t mean that they have to be selectively executed. TestNG can be customised. But that doesn’t mean TestNG needs to support all customisations. It merely needs to provide for ways in which a user should be able to customise it to their needs and work with it. The sample I shared is an example of applying those customisations without TestNG having to undergo changes.

I don’t see a need of constantly re-executing the tests from the base class. If it’s a matter of setup/teardown one could always use @Before/@after that would be re-executed every time before a child class from the base class.

To me that is just one way of looking at things. If a particular method shouldn’t be executed every time, then you should not be using the @Test annotation. You could very well use the configuration annotations such as @BeforeTest (which gets executed only once per <test> tag) or @BeforeSuite (which gets executed only once per <suite> tag) no? Wouldn’t they work for you.

I am not sure, if your test methods in your base class are really test methods. To me they sound more like configurations which establish the setup conditions to be satisfied for the test to run through.

@juherr - I am not quite sure I agree with the use case here. A test irrespective of where it is found is meant to be executed. If there is scenario wherein we don’t want it to be executed for all child classes, then we should be using the configuration annotations and not the @Test annotation.

On the implementation side, I think this will add a lot of chaos to the codebase especially when people start resorting to using groups as a means for execution.

Moreover, I feel that there is already a way of doing this currently within TestNG (which I have shared as an example). Why not just leverage that? This seems to be a one off use case, and doesn’t seem to fit into the regular way of how TestNG is used.