Parameterized Android tests on Robolectric could not be run

Description

I had the need to test a specific feature of a paging component that used Room, and I needed to use a real in memory version of Room, so I needed the Context. Since the tests cases needed to cover different inputs, but the logic was the same, I intended to use Parameterized tests from JUnit 4

Steps to Reproduce

Open the SimpleParameterizedTest.kt class and try to execute it with Robolectric.

Actual Results

If we run the suite with @RunWith(AndroidJUnit4::class), we get the following error:

java.lang.RuntimeException: Delegate runner 'org.robolectric.RobolectricTestRunner' for AndroidJUnit4 could not be loaded.

	at androidx.test.ext.junit.runners.AndroidJUnit4.throwInitializationError(AndroidJUnit4.java:92)
	at androidx.test.ext.junit.runners.AndroidJUnit4.loadRunner(AndroidJUnit4.java:82)
	at androidx.test.ext.junit.runners.AndroidJUnit4.loadRunner(AndroidJUnit4.java:51)
	at androidx.test.ext.junit.runners.AndroidJUnit4.<init>(AndroidJUnit4.java:46)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBuilder.java:104)
	at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(AnnotatedBuilder.java:86)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
	at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
	at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
	at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:33)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:49)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at androidx.test.ext.junit.runners.AndroidJUnit4.loadRunner(AndroidJUnit4.java:72)
	... 16 more
Caused by: org.junit.runners.model.InitializationError
	at org.junit.runners.ParentRunner.validate(ParentRunner.java:418)
	at org.junit.runners.ParentRunner.<init>(ParentRunner.java:84)
	at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:65)
	at org.robolectric.internal.SandboxTestRunner.<init>(SandboxTestRunner.java:56)
	at org.robolectric.RobolectricTestRunner.<init>(RobolectricTestRunner.java:92)
	... 21 more

If we run the suite with @RunWith(Parameterized::class), we instead get the following error:

java.lang.IllegalStateException: No instrumentation registered! Must run under a registering instrumentation.

	at androidx.test.platform.app.InstrumentationRegistry.getInstrumentation(InstrumentationRegistry.java:45)
	at androidx.test.core.app.ApplicationProvider.getApplicationContext(ApplicationProvider.java:41)
	at net.orgiu.tests.SimpleParameterizedTest.<init>(SimpleParameterizedTest.kt:15)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.junit.runners.parameterized.BlockJUnit4ClassRunnerWithParameters.createTestUsingConstructorInjection(BlockJUnit4ClassRunnerWithParameters.java:43)
	at org.junit.runners.parameterized.BlockJUnit4ClassRunnerWithParameters.createTest(BlockJUnit4ClassRunnerWithParameters.java:38)
	at org.junit.runners.BlockJUnit4ClassRunner$1.runReflectiveCall(BlockJUnit4ClassRunner.java:266)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.BlockJUnit4ClassRunner.methodBlock(BlockJUnit4ClassRunner.java:263)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

AndroidX Test and Android OS Versions

I am trying with the following dependencies

testImplementation 'androidx.test:runner:1.1.1'
testImplementation 'androidx.test.espresso:espresso-core:3.1.1'
testImplementation 'androidx.test.ext:junit:1.1.0'
testImplementation 'org.robolectric:robolectric:4.1'

Link to a public git repo demonstrating the problem:

GitHub Repository

[jongerrish]

Hey @tiwiz So, this is kind of tough right now…

• Robolectric needs its own test runner in order to create its own sandbox environment to execute Android code, so to write parameterized tests in Robolectric you need to use @RunWith(RobolectricParameterizedTestRunner.class)
• Instrumentation tests don’t have that restriction so you can legitimately use @RunWith(Parameterized.class)
• The unified test runner is AndroidJUnit4.class which will detect the environment in which you run and delegate to either Robolectric or Instrumentation under the covers. However, it just supports regular JUnit4 style tests, not parameterized.

Long term Nitrogen will solve this as we won’t need any custom JUnit4 test runners to bootstrap Robolectric or Instrumentation tests (which will be done via N2 itself.

So… what can we do in the meantime?

I chatted with the team and we think creating a ParameterizedAndroidJUnit4.class which will would replace RobolectricParameterizedTestRunner.class and also delegate to the regular Parameterized.class to work on-device too.

This helps with our goal of consolidating Robolectric test runners into AndroidX Test and it will also remain compatible (although not necessary) once Nitrogen + Sheep device are shipped.

What do you think about having a go at implementing this ParameterizedAndroidJUnit4 test runner? We’re not going to get to it right very soon, but we’d provide full support in guidance and code reviews and getting it into a release. You’d get all the glory 😃

[mrctrifork]

And here we are one year later 😢