org.junit.runners.model.InvalidTestClassError starting with Android Studio Dolphin | 2021.3.1

With previous version Android Studio 2021.2.1.14 it works fine, with the latest stable one every single time I try to run a local instrumented test class it throws the following exception:

org.junit.runners.model.InvalidTestClassError: Invalid test class 'x.y.z':
  1. No runnable methods
	at org.junit.runners.ParentRunner.validate(ParentRunner.java:525)
	at org.junit.runners.ParentRunner.<init>(ParentRunner.java:92)
	at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:74)
	at androidx.test.internal.runner.junit4.AndroidJUnit4ClassRunner.<init>(AndroidJUnit4ClassRunner.java:43)
	at androidx.test.internal.runner.junit4.AndroidJUnit4Builder.runnerForClass(AndroidJUnit4Builder.java:65)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
	at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:37)
	at androidx.test.internal.runner.AndroidRunnerBuilder.runnerForClass(AndroidRunnerBuilder.java:153)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:70)
	at androidx.test.internal.runner.TestLoader.doCreateRunner(TestLoader.java:73)
	at androidx.test.internal.runner.TestLoader.getRunnersFor(TestLoader.java:105)
	at androidx.test.internal.runner.TestRequestBuilder.build(TestRequestBuilder.java:804)
	at androidx.test.runner.AndroidJUnitRunner.buildRequest(AndroidJUnitRunner.java:613)
	at androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:411)
	at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:2248)

Tested also with beta & canary and got the same result. Android Studio -> Preferences -> Build, Execution, Deployment -> Testing -> Run Android Instrumented Tests using Gradle is UNCHECKED

Jupiter 5.9.0 de.mannodermaus.junit5:android-test-core:1.3.0 de.mannodermaus.junit5:android-test-runner:1.3.0 classpath(“de.mannodermaus.gradle.plugins:android-junit5:1.8.2.1”)

[mihaicristianpetrescu]

@mannodermaus Any thoughts on this one? It will start to fallback even further as more Android Studio releases go out + now we are restricted to using “com.android.tools.build:gradle:7.2.0” due to the older Android Studio version (2021.2.1.14).

[mihaicristianpetrescu]

I have tried to get more details from the runs but I haven’t been able to pull more data out of the instrumentation runs. If I remove the JUnit 5 from the project both Unit Tests & Instrumentations should fails since they are build atop the JUnit 5 and use a lot of features available only in JUnit 5, so it’s not an actual options.

Yeah I was curious to find out if there’s something that your plugin might be leaning on something specific from Android Studio and that’s why it needs to have that checkbox turned on now or in general and it was a “fluke” until the latest version that it worked without it checked.