Add a way to repeat all test suit

Currently, the JUnit has the option of using the @RepeatedTest to allow a test to run multiple times.

Using the JITWatch, we can see how the application is behaving, so using the test suit would be a great way to get this information. However, running the tests one time each we cannot give the JIT enough time to really optimize.

In summary, to achieve this, now I’m replacing all @Test() with @RepeatedTest(4500) and running the test suit via console, like:

mvn package -DskipTests
java -XX:+UnlockDiagnosticVMOptions -XX:+PrintAssembly -XX:+LogCompilation -XX:LogFile=mylogfile.log  -jar ~/Downloads/junit-platform-console-standalone-1.8.0-M1.jar --class-path "..." --scan-classpath

This way, after running the test suit, I get the “mylogfile.log” with a lot of JIT decisions. I’d like to know if would be of use to have an extension or an annotation that could be easy to add that could repeat all the tests instead of having to do a search and replace in the test suit.

[nipafx]

I recommend to open an issue with Junit 5 because adding an option --repeat n to the launcher sounds like it deserves a discussion (there seems to be some benefit and I assume it would be quite easy to implement).

I’ll go ahead and mark this issue as blocked until that discussion was resolved (or it was stated that it can’t be had there).

[nipafx]

Hi @Thihup, I’m not sure whether this is possible in general. Actually, I’d be surprised if it worked for all cases, but it may work for many (e.g. the @Test annotated ones).

That said, I think this would be a nice feature for junit-platform. To run the entire suite repeatedly for a fixed number of times or even until aborted. Have you checked whether junit-platform-console-standalone has such a parameter? Maybe @sormuras can help.