Groovy 3 Unit Test Class: TestNGException at Parameters.java:437

TestNG Version

Note: only the latest version is supported

7.3.0

Expected behavior

Test pass without exception

Actual behavior

Gradle suite > Gradle test > foo.FooTest > setMetaClass FAILED org.testng.TestNGException at Parameters.java:437 Received a failure event for test with unknown id ‘55.7’. Registered test ids: ‘[55.3, 55.2, 55.1, :test]’

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

Test case sample

Please, share the test case (as small as possible) which shows the issue

Java version AdoptOpenJDK (build 25.262-b10, mixed mode) Gradle version 6.6.1 Groovy version 3.0.5

Project attached: g3tng.tar.gz

src/test/groovy/foo/FooTest.groovy

package foo
import groovy.transform.CompileStatic
import org.testng.annotations.Test
@CompileStatic
@Test
class FooTest {
    void test() {
        assert true
    }
}

build.gradle.kts

plugins {
    groovy
}
repositories {
    jcenter()
}
dependencies {
    implementation("org.codehaus.groovy:groovy-all:3.0.5")
    testImplementation("org.testng:testng:7.3.0")
}
tasks.test {
    useTestNG()
}

Observations:

1. No issue with groovy version 2.x

2. TestNG selects setMetaClass and setProperty as test methods. When runs with ReportNG:

org.testng.TestNGException:
Cannot inject @Test annotated Method [setMetaClass] with [interface groovy.lang.MetaClass].

org.testng.TestNGException:
Cannot inject @Test annotated Method [setProperty] with [class java.lang.String, class java.lang.Object].

[ctzen]

I believe the Groovy project people can explain it better than me, perhaps you can ask in that JIRA.

However, this is my understanding of the issue.

Groovy augment groovy classes with internal methods when converting them to Java classes. 2 of such augmented methods are setMetaClass() and setProperty().

Before Groovy 3, these internal methods were marked as synthetic, which TestNG would ignore when discovering test methods. (I traced the TestNG method discovery to AnnotationHelper.findMethodsWithAnnotation())

Since Groovy 3, these internal methods are no longer marked as synthetic, and TestNG will pick them up as test methods. These internal methods are now annotated with @groovy.transform.Internal

The fix is for TestNG to NOT pick these Groovy internal methods up as test methods.

The straight forward way is of course to ignored methods annotated with @groovy.transform.Internal. However, wouldn’t that means Groovy will become a transitional dependency of TestNG?

Another possible way is to enhance the TestNG test method discovery. The 2 erroneous methods both have parameters but no DataProvider. If TestNG is to ignore methods with parameters but no DataProvider, that should also solve this issue.

May be we can ask the JACOCO project how they did it?

[juherr]

For the record, some related Jacoco issues: https://github.com/jacoco/jacoco/pull/610 https://github.com/jacoco/jacoco/pull/733