Activity fails to launch with ActivityTestRule

Description

We have a set of UI tests (~150) that are being run as part of CI and we use ActivityTestRule to start activities on connected emulator. Lately we noticed some of the tests failing irregularly (maybe 50% of time), but only on our 2 CI machines (both of them so it is not local to single machine). Locally they seem to always pass.

Most of the tests use following structure, launching activity manually because some setup is needed:

    @get:Rule
    val activityRule = ActivityTestRule(ActivityUnderTest::class.java, true, false)

    @Test
    fun exampleTest() {
        setupState()

        activityRule.launchActivity(Intent().apply { putExtras() })

        executeTestAndVerifyResults()
    }

The tests that fail (~13) are always the same and they fail with: RuntimeException: Could not launch intent Intent { flg=0x10000000 cmp=package/FailingActivity (has extras) } within 45 seconds. However, when I inspected the thread dump, I don’t see anything holding up tha main thread:

02-14 13:45:21.245  4306  4319 E THREAD_STATE:   Thread[Instr: com.example.CustomRunner,5,main]
02-14 13:45:21.245  4306  4319 E THREAD_STATE:     dalvik.system.VMStack.getThreadStackTrace(Native Method)
02-14 13:45:21.245  4306  4319 E THREAD_STATE:     java.lang.Thread.getStackTrace(Thread.java:580)
02-14 13:45:21.245  4306  4319 E THREAD_STATE:     java.lang.Thread.getAllStackTraces(Thread.java:522)
02-14 13:45:21.245  4306  4319 E THREAD_STATE:     androidx.test.runner.MonitoringInstrumentation.getThreadState(MonitoringInstrumentation.java:643)
02-14 13:45:21.245  4306  4319 E THREAD_STATE:     com.example.CustomRunner.dumpThreadStateToOutputs(GeorgeTestRunner.kt:15)
02-14 13:45:21.245  4306  4319 E THREAD_STATE:     androidx.test.runner.MonitoringInstrumentation.startActivitySync(MonitoringInstrumentation.java:450)
02-14 13:45:21.245  4306  4319 E THREAD_STATE:     com.example.CustomRunner.startActivitySync(GeorgeTestRunner.kt:21)
02-14 13:45:21.245  4306  4319 E THREAD_STATE:     androidx.test.rule.ActivityTestRule.launchActivity(ActivityTestRule.java:358)
02-14 13:45:21.245  4306  4319 E THREAD_STATE:     com.example.ExampleTest.failingTest(PromotionAttachedToSectionTests.kt:108)
02-14 13:45:21.245  4306  4319 E THREAD_STATE:     java.lang.reflect.Method.invoke(Native Method)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     androidx.test.internal.runner.junit4.statement.RunBefores.evaluate(RunBefores.java:80)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     androidx.test.rule.ActivityTestRule$ActivityStatement.evaluate(ActivityTestRule.java:531)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.rules.RunRules.evaluate(RunRules.java:20)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.ParentRunner.run(ParentRunner.java:363)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.Suite.runChild(Suite.java:128)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.Suite.runChild(Suite.java:27)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runners.ParentRunner.run(ParentRunner.java:363)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runner.JUnitCore.run(JUnitCore.java:137)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     org.junit.runner.JUnitCore.run(JUnitCore.java:115)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     androidx.test.internal.runner.TestExecutor.execute(TestExecutor.java:56)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     androidx.test.runner.AndroidJUnitRunner.onStart(AndroidJUnitRunner.java:388)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1879)
...
02-14 13:45:21.246  4306  4319 E THREAD_STATE:   Thread[main,5,main]
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     android.os.MessageQueue.nativePollOnce(Native Method)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     android.os.MessageQueue.next(MessageQueue.java:323)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     android.os.Looper.loop(Looper.java:135)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     android.app.ActivityThread.main(ActivityThread.java:5417)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     java.lang.reflect.Method.invoke(Native Method)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
...
02-14 13:45:21.246  4306  4319 E THREAD_STATE:   Thread[MonitoringInstrumentation,5,main]
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     java.lang.Object.wait(Native Method)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     android.app.Instrumentation.startActivitySync(Instrumentation.java:409)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     androidx.test.runner.MonitoringInstrumentation.access$101(MonitoringInstrumentation.java:98)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     androidx.test.runner.MonitoringInstrumentation$4.call(MonitoringInstrumentation.java:443)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     androidx.test.runner.MonitoringInstrumentation$4.call(MonitoringInstrumentation.java:440)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     java.util.concurrent.FutureTask.run(FutureTask.java:237)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)
02-14 13:45:21.246  4306  4319 E THREAD_STATE:     java.lang.Thread.run(Thread.java:818)

The weird thing (at least to me) is that the failing tests are all trying to launch same activity. But this activity is not special in any way that I can see. It has no special launch mode defined in the manifest. And similar activities with same tests structure run fine. Also it does not seem to be influenced by previous test, as it fails even if I suppress previous test and on the other hand, if I suppress the failing one, the next test runs fine. Not only is the onCreate of the activity not called, but when i add an empty constructor to the activity, even that is not called.

In the meantime, the main thread seems to be totally idle, as I tried to periodically post a ping from previous test:

    private fun startMainThreadPings() {
        val start = DateTime.now()
        val looper = Looper.getMainLooper()
        looper.setMessageLogging {
            Log.d("FERI", "looper: $it")
        }

        looper.queue.addIdleHandler {
            Log.d("FERI", "queue is idle")
            true
        }

        val h = Handler(looper)
        var i = 0
        val ping = object : Runnable {
            override fun run() {
                Log.d("FERI", "ping: ${i++}")
                val instance = ActivityLifecycleMonitorRegistry.getInstance()
                Stage.values().forEach {
                    val activitiesInStage = instance.getActivitiesInStage(it)
                    if (activitiesInStage.isNotEmpty()) Log.d("FERI", "$it -> $activitiesInStage")
                }

                if (start.plus(Minutes.ONE).isAfterNow) h.postDelayed(this, 50)
            }
        }

        h.post(ping)
    }

The logs are printed periodically, both the “ping: N” and “queue is idle”. I also tried capturing a trace for the profiler (which was a pain as it only reproduces on remote builder), but it did only show the same thing: idle main thread and test waiting to be launched, eventually timing out.

I am pretty desperate, so if you have an idea I could try or need any more info, let me know. Thanks

AndroidX Test and Android OS Versions

AndroidX Test - 1.1.1 Android - 6.0 emulator - 28.0.23 image - system-images/android-23/google_apis/x86 host - linux

[yuuki3655]

Yes, the log of the failed test run shows that ActivityManager found your FailingActivity was already in the task stack so the manager just brought it foreground instead of creating a new one. It’s likely that the FailingActivity was already in resumed state so none of onCreate, onStart, onResumed weren’t called.

Could you try adding FLAG_ACTIVITY_CLEAR_TASK to your intent or try using ActivityScenario? ActivityTestRule tries to finish testing Activity after each test case but it can give up after certain timeout. This could happen for variety of reasons by the android framework.

[yaxi]

Sorry to bring this old thread up. I’m currently using createAndroidComposeRule which is using ActivityScenarioRule behind the scene and still getting random errors on the Firebase TestLab.

When checking the log, it has the similar pattern that https://github.com/android/android-test/issues/211#issuecomment-729899072 shows.

Tried with:

    @After
    fun breakDown() {
        composeTestRule.activityRule.scenario.moveToState(Lifecycle.State.DESTROYED)
    }

and then

    companion object {
        @AfterClass
        @JvmStatic
        fun teardown() {
            Thread.sleep(500)
        }
    }

either or both seemed not working. The tests will fail at one individual test case, however all of the test cases are working perfectly fine on either emulator or physical device.

May I ask how should I proceed to make it less flaky? thanks.