JUnit tests not executed or failures with Junit 5 & Surefire >= 3.0.0-M4

Describe the bug With the latest versions of the surefire plugin, no features are executed, or the run fails entirely.

To Reproduce Steps to reproduce the behavior: Use the example from: https://github.com/bonigarcia/mastering-junit5/tree/b19313dc1f6a1b7c4421e84ac9d6afd117c218d5/junit5-cucumber Run it once, everything works fine (surefire 2.22.2):

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0

In the POM, change surefire to 3.0.0-M3, all still good:

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0

In the POM, change surefire to 3.0.0-M4, tests are executed (can check with stdout ‘ADD’ in steps), but not reported:

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
ADD
ADD
ADD
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

In the POM, change surefire to 3.0.0-M5, tests are executed (can check with stdout ‘ADD’ in steps), but not reported, and an error is logged:

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[ERROR] jul 06, 2021 9:47:26 AM org.junit.platform.launcher.core.LauncherConfigurationParameters loadClasspathResource
[ERROR] INFO: Loading JUnit Platform configuration parameters from classpath resource [file:/C:/workspace/IdeaProjects/junit5-cucumber/target/test-classes/junit-platform.properties].
ADD
ADD
ADD
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

Context & Motivation We just want to use Junit5 + Cucumber, but with the latest versions of everything it just doesn’t work. Since the results are not reported to surefire, it also doesn’t fail on errors, so we don’t notice if there’s a problem.

Your Environment

• Cucmber 6.10.4
• Junit-Jupiter 5.7.2
• Windows 10
• Maven 3.8.1

[mpkorstanje]

With Cucumber v7 (not yet released) and JUnit 5.8 (not yet released) we will deprecate @Cucumber in favour of @Suite. This should coincidently fix the issue with Surefire > M4.

Add:

<dependency>
   <groupId>org.junit.platform</groupId>
   <artifactId>junit-platform-suite</artifactId>
   <version>${junit-platform.version}</version>
   <scope>test</scope>
</dependency>

Then use:

package com.example;

import org.junit.platform.suite.api.ConfigurationParameter;
import org.junit.platform.suite.api.SelectClasspathResource;
import org.junit.platform.suite.api.Suite;

import static io.cucumber.junit.platform.engine.Constants.GLUE_PROPERTY_NAME;

@Suite
@SelectClasspathResource("com/exmaple")
@ConfigurationParameter(key = GLUE_PROPERTY_NAME, value = "com.example")
public class RunCucumberTest {
}

To improve the readability of the reports provide the cucumber.junit-platform.naming-strategy=long configuration parameter.

[mpkorstanje]

This is not a problem with Cucumber. Please do not report additional instances. Do upvote the issue in the Surefire bug tracker:

https://issues.apache.org/jira/browse/SUREFIRE-1849

edit: @aSemy fixed the mistake in message. Cheers.