Video recording causes emulators to disconnect

Describe the bug If video recording is turned on for failed tests, emulators will randomly disconnect after a test completes, right before Marathon deletes the video. The emulator disconnects in a way that sometimes my next automated step of killing the emulator times out, when it normally takes <1s to close all three. It’s consistent in that one or two emulators always fail when running about 250 tests, meaning the remaining emulators have to pick up the slack and then the test run takes significantly longer.

My API 28 emulator runs were fine but we don’t want to run on 28 anymore. The only thing that makes this not happen is to turn off video recording and only use screenshots. Coincidentally it doesn’t actually save screenshots, but I do have a bunch of 1 byte gifs. This is a separate issue, but if you know a quick fix I’d like to hear it.

I’ve tried a different setup/system images/skins for an API 30 emulator, increasing ram and heap, turning off headless emulator runs, using the default video config instead of a custom one (the height was wrong for a while but fixing it did nothing), removing video config (dimensions, bitrate), running only a single test 100 times (sometimes I even got all three emulators to fail!), upgrading Marathon to an official release, getting the newest SDK tools, and a number of other things.

To Reproduce Steps to reproduce the behaviour: Tried on both a version of Marathon between 0.6.5 and 0.7.0, as well as the newest, 0.7.1.

name: "SNIP Tests"
outputDir: "build/reports/marathon"
debug: true
isCodeCoverageEnabled: true
vendorConfiguration:
  type: "Android"
  screenRecordConfiguration:
    preferableRecorderType: "video"
    videoConfiguration:
      enabled: true
      timeLimit: 1000
  applicationApk: "snip/build/outputs/apk/internal/debug/snip.apk"
  testApplicationApk: "snip/build/outputs/apk/androidTest/internal/debug/snip.apk"
  autoGrantPermission: true
  instrumentationArgs:
    debug: "false"
  applicationPmClear: true
  testApplicationPmClear: true
  vendor: ADAM
  waitForDevicesTimeoutMillis: 60000
  allureConfiguration:
    enabled: true
  fileSyncConfiguration:
    pull:
      - relativePath: "screenshots"
        aggregationMode: POOL
deviceInitializationTimeoutMillis: 180000
fallbackToScreenshots: true
filteringConfiguration:
  allowlist:
    - type: "fully-qualified-class-name"
      regex: ".*"
    - type: "annotation"
      regex: ".*"
    - type: "package"
      regex: ".*"
  blocklist:
    - type: "annotation"
      regex: "org.junit.Ignore"
testClassRegexes:
  - ".*"
retryStrategy:
  type: "fixed-quota"
  totalAllowedRetryQuota: 100
  retryPerTestQuota: 3
batchingStrategy:
  type: "fixed-size"
  size: 10
  durationMillis: 200000
  percentile: 80.0
  timeLimit: "-PT1H"
  lastMileLength: 10

Emulator config.ini:

AvdId = Pixel_5_API_30
PlayStore.enabled = true
abi.type = x86_64
avd.ini.displayname = Pixel 5 API 30
avd.ini.encoding = UTF-8
disk.dataPartition.size = 6442450944
fastboot.chosenSnapshotFile = 
fastboot.forceChosenSnapshotBoot = no
fastboot.forceColdBoot = no
fastboot.forceFastBoot = yes
hw.accelerometer = yes
hw.arc = false
hw.audioInput = yes
hw.battery = yes
hw.camera.back = virtualscene
hw.camera.front = emulated
hw.cpu.arch = x86_64
hw.cpu.ncore = 4
hw.dPad = no
hw.device.hash2 = MD5:3274126e0242a0d86339850416b0ce34
hw.device.manufacturer = Google
hw.device.name = pixel_5
hw.gps = yes
hw.gpu.enabled = yes
hw.gpu.mode = auto
hw.initialOrientation = Portrait
hw.keyboard = yes
hw.lcd.density = 440
hw.lcd.height = 2340
hw.lcd.width = 1080
hw.mainKeys = no
hw.ramSize = 1536
hw.sdCard = yes
hw.sensors.orientation = yes
hw.sensors.proximity = yes
hw.trackBall = no
image.sysdir.1 = system-images/android-30/google_apis/x86_64/
runtime.network.latency = none
runtime.network.speed = full
sdcard.size = 512M
showDeviceFrame = yes
skin.dynamic = yes
tag.display = Google APIs
tag.id = google_apis
vm.heapSize = 256

Logs and reports Classes, packages, and test names censored.

Common style of run where two emulators fail:

5558’s logs:

21:40:48.1564730Z I 17:40:48.156 [AndroidDevice - execution - 127.0.0.1:5037:emulator-5558-1 @coroutine#3665] <DebugTestRunListener> testEnded 127.0.0.1:5037:emulator-5558 test = TestIdentifier(className=com.some.package.TestLastSuccessfulClass, testName=testLastSuccessfulCompleteTest)
21:40:48.1568380Z 75% | [omni]-[127.0.0.1:5037:emulator-5558] com.some.package.TestLastSuccessfulClass#testLastSuccessfulCompleteTest ended
21:40:48.1572560Z I 17:40:48.156 [AndroidDevice - execution - 127.0.0.1:5037:emulator-5558-1 @coroutine#3665] <DebugTestRunListener> testStarted 127.0.0.1:5037:emulator-5558 test = TestIdentifier(className=com.some.package.ClassFailing, testName=testLastTestBeforeDisconnect)
21:40:48.1577070Z 75% | [omni]-[127.0.0.1:5037:emulator-5558] com.some.package.ClassFailing#testLastTestBeforeDisconnect started
21:40:50.4708670Z D 17:40:50.470 [main @coroutine#75] <Scheduler> device 127.0.0.1:5037:emulator-5558 disconnected
21:40:50.4710520Z D 17:40:50.470 [main @coroutine#76] <DevicePoolActor[omni]> remove device 127.0.0.1:5037:emulator-5558
21:40:50.4711690Z D 17:40:50.470 [main @coroutine#76] <DevicePoolActor[omni]> devices.size = 1
21:40:50.4712730Z D 17:40:50.470 [DeviceMonitor @coroutine#6] <AdamDeviceProvider> Device emulator-5558 changed state to DISCONNECTED
21:40:50.4713980Z D 17:40:50.470 [main @coroutine#3920] <D.0.0.1:5037:emulator-5558]> terminate 127.0.0.1:5037:emulator-5558
21:40:50.4715550Z E 17:40:50.470 [AndroidDevice - execution - 127.0.0.1:5037:emulator-5558-1 @coroutine#3913] <AndroidDevice> Unable to start screenrecord
21:40:50.4717240Z com.malinskiy.adam.exception.RequestRejectedException: No exit code delimiter found in 
21:40:50.4718080Z 	at com.malinskiy.adam.request.shell.v1.ShellResultResponseTransformer.transform(ShellResultResponseTransformer.kt:35)
21:40:50.4718930Z 	at com.malinskiy.adam.request.shell.v1.SyncShellCommandRequest.transform(SyncShellCommandRequest.kt:37)
21:40:50.4719780Z 	at com.malinskiy.adam.request.SynchronousRequest.readElement$suspendImpl(SynchronousRequest.kt:43)
21:40:50.4720750Z 	at com.malinskiy.adam.request.SynchronousRequest$readElement$1.invokeSuspend(SynchronousRequest.kt)
21:40:50.4721750Z 	at (Coroutine boundary.()
21:40:50.4722190Z 	at kotlinx.coroutines.TimeoutKt.withTimeoutOrNull(Timeout.kt:101)
21:40:50.4722700Z 	at com.malinskiy.marathon.android.adam.AdamAndroidDevice.safeStartScreenRecorder(AdamAndroidDevice.kt:316)
21:40:50.4723820Z 	at com.malinskiy.marathon.android.executor.listeners.video.ScreenRecorder.startRecordingTestVideo(ScreenRecorder.kt:27)
21:40:50.4724940Z 	at com.malinskiy.marathon.android.executor.listeners.video.ScreenRecorder.run(ScreenRecorder.kt:17)
21:40:50.4726090Z 	at com.malinskiy.marathon.android.executor.listeners.video.ScreenRecorderTestBatchListener$testStarted$2.invokeSuspend(ScreenRecorderTestBatchListener.kt:61)
21:40:50.4727160Z Caused by: com.malinskiy.adam.exception.RequestRejectedException: No exit code delimiter found in 
21:40:50.4728200Z 	at com.malinskiy.adam.request.shell.v1.ShellResultResponseTransformer.transform(ShellResultResponseTransformer.kt:35)
21:40:50.4729570Z 	at com.malinskiy.adam.request.shell.v1.SyncShellCommandRequest.transform(SyncShellCommandRequest.kt:37)
21:40:50.4730570Z 	at com.malinskiy.adam.request.SynchronousRequest.readElement$suspendImpl(SynchronousRequest.kt:43)
21:40:50.4731510Z 	at com.malinskiy.adam.request.SynchronousRequest$readElement$1.invokeSuspend(SynchronousRequest.kt)
21:40:50.4732350Z 	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
21:40:50.4733200Z 	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
21:40:50.4733930Z 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
21:40:50.4734710Z 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
21:40:50.4735740Z 	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
21:40:50.4736710Z 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
21:40:50.4737470Z 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
21:40:50.4738100Z 	at java.base/java.lang.Thread.run(Thread.java:829)
21:40:50.4740490Z D 17:40:50.470 [AndroidDevice - execution - 127.0.0.1:5037:emulator-5558-1 @coroutine#3913] <ScreenRecorder> Recording finished in 2314ms /sdcard/com.some.package.ClassFailing-testLastTestBeforeDisconnect-5a2e0827-d1d9-48b1-ba52-9d603a226924.mp4
21:40:50.4742350Z W 17:40:50.471 [main @coroutine#3664] <D.0.0.1:5037:emulator-5558]> Device execution has been cancelled
21:40:50.4743280Z kotlinx.coroutines.JobCancellationException: DeferredCoroutine was cancelled
21:40:50.4743740Z 	at kotlinx.coroutines.JobSupport.cancel(JobSupport.kt:1578)
21:40:50.4744340Z 	at kotlinx.coroutines.Job$DefaultImpls.cancel$default(Job.kt:183)
21:40:50.4744800Z 	at com.malinskiy.marathon.execution.device.DeviceActor.terminate(DeviceActor.kt:228)
21:40:50.4745840Z 	at com.malinskiy.marathon.execution.device.DeviceActor.access$terminate(DeviceActor.kt:27)
21:40:50.4747110Z 	at com.malinskiy.marathon.execution.device.DeviceActor$state$1$6$3.invoke(DeviceActor.kt:118)
21:40:50.4748480Z 	at com.malinskiy.marathon.execution.device.DeviceActor$state$1$6$3.invoke(DeviceActor.kt:117)
21:40:50.4749360Z 	at kotlinx.coroutines.InvokeOnCompletion.invoke(JobSupport.kt:1391)
21:40:50.4750220Z 	at kotlinx.coroutines.JobSupport.completeStateFinalization(JobSupport.kt:318)
21:40:50.4751050Z 	at kotlinx.coroutines.JobSupport.tryFinalizeSimpleState(JobSupport.kt:295)
21:40:50.4751830Z 	at kotlinx.coroutines.JobSupport.tryMakeCompleting(JobSupport.kt:856)
21:40:50.4752470Z 	at kotlinx.coroutines.JobSupport.makeCompletingOnce$kotlinx_coroutines_core(JobSupport.kt:828)
21:40:50.4752940Z 	at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:100)
21:40:50.4753980Z 	at (Coroutine boundary.()
21:40:50.4754560Z 	at com.malinskiy.marathon.android.adam.AdamAndroidDevice.execute(AdamAndroidDevice.kt:361)
21:40:50.4755460Z 	at com.malinskiy.marathon.execution.device.DeviceActor$executeBatch$2.invokeSuspend(DeviceActor.kt:187)
21:40:50.4755910Z Caused by: kotlinx.coroutines.JobCancellationException: DeferredCoroutine was cancelled
21:40:50.4756720Z 	at kotlinx.coroutines.JobSupport.cancel(JobSupport.kt:1578)
21:40:50.4757490Z 	at kotlinx.coroutines.Job$DefaultImpls.cancel$default(Job.kt:183)
21:40:50.4758140Z 	at com.malinskiy.marathon.execution.device.DeviceActor.terminate(DeviceActor.kt:228)
21:40:50.4758730Z 	at com.malinskiy.marathon.execution.device.DeviceActor.access$terminate(DeviceActor.kt:27)
21:40:50.4759200Z 	at com.malinskiy.marathon.execution.device.DeviceActor$state$1$6$3.invoke(DeviceActor.kt:118)
21:40:50.4760000Z 	at com.malinskiy.marathon.execution.device.DeviceActor$state$1$6$3.invoke(DeviceActor.kt:117)
21:40:50.4760690Z 	at kotlinx.coroutines.InvokeOnCompletion.invoke(JobSupport.kt:1391)
21:40:50.4761190Z 	at kotlinx.coroutines.JobSupport.completeStateFinalization(JobSupport.kt:318)
21:40:50.4761850Z 	at kotlinx.coroutines.JobSupport.tryFinalizeSimpleState(JobSupport.kt:295)
21:40:50.4762610Z 	at kotlinx.coroutines.JobSupport.tryMakeCompleting(JobSupport.kt:856)
21:40:50.4763170Z 	at kotlinx.coroutines.JobSupport.makeCompletingOnce$kotlinx_coroutines_core(JobSupport.kt:828)
21:40:50.4763890Z 	at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:100)
21:40:50.4764770Z 	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
21:40:50.4765490Z 	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
21:40:50.4766010Z 	at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
21:40:50.4766410Z 	at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
21:40:50.4767210Z 	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
21:40:50.4767920Z 	at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
21:40:50.4769120Z 	at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
21:40:50.4769940Z 	at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
21:40:50.4770870Z 	at com.malinskiy.marathon.Marathon.run(Marathon.kt:52)
21:40:50.4771600Z 	at com.malinskiy.marathon.cli.ApplicationViewKt$main$1.invoke(ApplicationView.kt:62)
21:40:50.4772270Z 	at com.malinskiy.marathon.cli.ApplicationViewKt$main$1.invoke(ApplicationView.kt:29)
21:40:50.4773080Z 	at com.xenomachina.argparser.SystemExitExceptionKt.mainBody(SystemExitException.kt:74)
21:40:50.4773730Z 	at com.xenomachina.argparser.SystemExitExceptionKt.mainBody$default(SystemExitException.kt:72)
21:40:50.4774340Z 	at com.malinskiy.marathon.cli.ApplicationViewKt.main(ApplicationView.kt:27)
21:40:50.4776230Z E 17:40:50.471 [main @coroutine#3664] <D.0.0.1:5037:emulator-5558]> Invalid transition from DeviceState.Terminated event DeviceEvent.Terminate
21:40:50.4777810Z D 17:40:50.471 [main @coroutine#77] <QueueActor[DevicePoolId(name=omni)]> onReturnBatch 127.0.0.1:5037:emulator-5558
21:40:50.4778890Z E 17:40:50.471 [DefaultDispatcher-worker-3 @coroutine#3665] <AndroidDevice> 
21:40:50.4779490Z java.util.concurrent.CancellationException: The task was rejected
21:40:50.4780340Z 	at (Coroutine boundary.()
21:40:50.4781000Z 	at com.malinskiy.adam.transport.vertx.VertxSocketFactory.tcp(VertxSocketFactory.kt:38)
21:40:50.4781630Z 	at com.malinskiy.adam.AndroidDebugBridgeClient.execute(AndroidDebugBridgeClient.kt:52)
21:40:50.4782330Z 	at com.malinskiy.marathon.android.adam.AdamAndroidDevice.executeShellCommand(AdamAndroidDevice.kt:119)
21:40:50.4783570Z 	at com.malinskiy.marathon.android.executor.listeners.video.ScreenRecorderTestBatchListenerKt$verifyHealthy$2.invokeSuspend(ScreenRecorderTestBatchListener.kt:155)
21:40:50.4784600Z 	at kotlinx.coroutines.TimeoutKt.withTimeoutOrNull(Timeout.kt:101)
21:40:50.4785540Z 	at com.malinskiy.marathon.android.executor.listeners.video.ScreenRecorderTestBatchListenerKt.verifyHealthy(ScreenRecorderTestBatchListener.kt:154)
21:40:50.4786720Z 	at com.malinskiy.marathon.android.executor.listeners.video.ScreenRecorderTestBatchListener.testRunFailed(ScreenRecorderTestBatchListener.kt:81)
21:40:50.4787800Z 	at com.malinskiy.marathon.android.executor.listeners.CompositeTestRunListener.testRunFailed(CompositeTestRunListener.kt:44)
21:40:50.4788640Z 	at com.malinskiy.marathon.android.adam.AndroidDeviceTestRunner.processEvents(AndroidDeviceTestRunner.kt:117)
21:40:50.4789660Z 	at com.malinskiy.marathon.android.adam.AndroidDeviceTestRunner$execute$2$1.invokeSuspend(AndroidDeviceTestRunner.kt:84)
21:40:50.4790720Z 	at kotlinx.coroutines.TimeoutKt.withTimeoutOrNull(Timeout.kt:101)
21:40:50.4791650Z 	at com.malinskiy.marathon.android.adam.AndroidDeviceTestRunner.execute(AndroidDeviceTestRunner.kt:67)
21:40:50.4792740Z 	at com.malinskiy.marathon.android.adam.AdamAndroidDevice$execute$2$1.invokeSuspend(AdamAndroidDevice.kt:359)
21:40:50.4793720Z 	at com.malinskiy.marathon.android.adam.AdamAndroidDevice$execute$2.invokeSuspend(AdamAndroidDevice.kt:357)
21:40:50.4794480Z Caused by: java.util.concurrent.CancellationException: The task was rejected
21:40:50.4795090Z 	at kotlinx.coroutines.ExceptionsKt.CancellationException(Exceptions.kt:22)
21:40:50.4795910Z 	at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.cancelJobOnRejection(Executors.kt:166)
21:40:50.4796760Z 	at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:128)
21:40:50.4797480Z 	at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:159)
21:40:50.4798310Z 	at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:397)
21:40:50.4799610Z 	at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:431)
21:40:50.4800470Z 	at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl$default(CancellableContinuationImpl.kt:420)
21:40:50.4801300Z 	at kotlinx.coroutines.CancellableContinuationImpl.resumeWith(CancellableContinuationImpl.kt:328)
21:40:50.4801950Z 	at io.vertx.kotlin.coroutines.VertxCoroutineKt$await$2$1.handle(VertxCoroutine.kt:139)
21:40:50.4802660Z 	at io.vertx.kotlin.coroutines.VertxCoroutineKt$await$2$1.handle(VertxCoroutine.kt:138)
21:40:50.4803290Z 	at io.vertx.core.impl.future.FutureImpl$3.onSuccess(FutureImpl.java:141)
21:40:50.4803870Z 	at io.vertx.core.impl.future.FutureBase.emitSuccess(FutureBase.java:60)
21:40:50.4804680Z 	at io.vertx.core.impl.future.FutureImpl.tryComplete(FutureImpl.java:211)
21:40:50.4805550Z 	at io.vertx.core.impl.future.Mapping.onSuccess(Mapping.java:40)
21:40:50.4806350Z 	at io.vertx.core.impl.future.FutureBase.lambda$emitSuccess$0(FutureBase.java:54)
21:40:50.4807310Z 	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
21:40:50.4808060Z 	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
21:40:50.4808690Z 	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
21:40:50.4809300Z 	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
21:40:50.4809940Z 	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
21:40:50.4810600Z 	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
21:40:50.4810970Z 	at java.base/java.lang.Thread.run(Thread.java:829)
21:40:50.4812280Z Caused by: java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@73b44219[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@303e8df[Wrapped task = CancellableContinuation(DispatchedContinuation[java.util.concurrent.ScheduledThreadPoolExecutor@ed0048d[Shutting down, pool size = 2, active threads = 0, queued tasks = 2, completed tasks = 11537], Continuation at com.malinskiy.adam.transport.vertx.VertxSocketFactory.tcp(VertxSocketFactory.kt:38)@778f330e]){Completed}@6fed1a2b]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@ed0048d[Shutting down, pool size = 2, active threads = 0, queued tasks = 2, completed tasks = 11537]
21:40:50.4813850Z 	at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2055)
21:40:50.4814730Z 	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:825)
21:40:50.4815800Z 	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340)
21:40:50.4816960Z 	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562)
21:40:50.4818060Z 	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:705)
21:40:50.4819060Z 	at kotlinx.coroutines.ExecutorCoroutineDispatcherImpl.dispatch(Executors.kt:125)
21:40:50.4819690Z 	... 19 common frames omitted
21:40:50.4822780Z W 17:40:50.472 [DefaultDispatcher-worker-3 @coroutine#3665] <TestRunResultsListener> uncompleted = com.some.package.ClassFailing#testLastTestBeforeDisconnect, 127.0.0.1:5037:emulator-5558
21:40:50.4825020Z W 17:40:50.472 [DefaultDispatcher-worker-3 @coroutine#3665] <TestRunResultsListener> uncompleted = com.some.package.MakePaymentPayPal#testProperAlertDisplaysWhenPayPalPayPaymentCallFailureOccurs, 127.0.0.1:5037:emulator-5558
21:40:50.4826870Z W 17:40:50.472 [DefaultDispatcher-worker-3 @coroutine#3665] <TestRunResultsListener> uncompleted = com.some.package.MakePaymentPayPal#testProperAlertDisplaysWhenPayPalPayBraintreeFailureOccurs, 127.0.0.1:5037:emulator-5558
21:40:50.4828110Z D 17:40:50.473 [DefaultDispatcher-worker-3 @coroutine#3665] <FileSyncTestRunListener> Pulling into /Applications/runner/_work/1/s/build/reports/marathon/device-files/omni/screenshots
21:43:48.1599230Z Root{application-window-token=android.view.ViewRootImpl$W@dfee4f4, window-token=android.view.ViewRootImpl$W@dfee4f4, has-window-focus=false, layout-params-type=1, layout-params-string={(0,0)(fillxfill) ty=BASE_APPLICATION wanim=0x10302fe
21:43:48.1600340Z   fl=LAYOUT_IN_SCREEN LAYOUT_INSET_DECOR SPLIT_TOUCH HARDWARE_ACCELERATED DRAWS_SYSTEM_BAR_BACKGROUNDS
21:43:48.1600860Z   pfl=FORCE_DRAW_STATUS_BAR_BACKGROUND FIT_INSETS_CONTROLLED
21:43:48.1602770Z   fitSides=}, decor-view-string=DecorView{id=-1, visibility=VISIBLE, width=1080, height=2340, has-focus=false, has-focusable=true, has-window-focus=false, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params={(0,0)(fillxfill) ty=BASE_APPLICATION wanim=0x10302fe
21:43:48.1604970Z   fl=LAYOUT_IN_SCREEN LAYOUT_INSET_DECOR SPLIT_TOUCH HARDWARE_ACCELERATED DRAWS_SYSTEM_BAR_BACKGROUNDS
21:43:48.1605480Z   pfl=FORCE_DRAW_STATUS_BAR_BACKGROUND FIT_INSETS_CONTROLLED
21:43:48.1606390Z   fitSides=}, tag=null, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=3}}
21:43:48.1606990Z 	at androidx.test.espresso.base.RootViewPicker.waitForRootToBeReady(RootViewPicker.java:69)
21:43:48.1607560Z 	at androidx.test.espresso.base.RootViewPicker.pickRootView(RootViewPicker.java:37)
21:43:48.1608090Z 	at androidx.test.espresso.base.RootViewPicker.get(RootViewPicker.java:16)
21:43:48.1608910Z 	at androidx.test.espresso.ViewInteractionModule.provideRootView(ViewInteractionModule.java:9)
21:43:48.1609610Z 	at androidx.test.espresso.ViewInteractionModule_ProvideRootViewFactory.provideRootView(ViewInteractionModule_ProvideRootViewFactory.java:8)
21:43:48.1610680Z 	at androidx.test.espresso.ViewInteractionModule_ProvideRootViewFactory.get(ViewInteractionModule_ProvideRootViewFactory.java:6)
21:43:48.1611640Z 	at androidx.test.espresso.ViewInteractionModule_ProvideRootViewFactory.get(ViewInteractionModule_ProvideRootViewFactory.java:7)
21:43:48.1612470Z 	at androidx.test.espresso.base.ViewFinderImpl.getView(ViewFinderImpl.java:12)
21:43:48.1613030Z 	at androidx.test.espresso.ViewInteraction$2.call(ViewInteraction.java:4)
21:43:48.1613710Z 	at androidx.test.espresso.ViewInteraction$2.call(ViewInteraction.java:2)
21:43:48.1614320Z 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
21:43:48.1615230Z 	at android.os.Handler.handleCallback(Handler.java:938)
21:43:48.1615730Z 	at android.os.Handler.dispatchMessage(Handler.java:99)
21:43:48.1616130Z 	at android.os.Looper.loop(Looper.java:223)
21:43:48.1616540Z 	at android.app.ActivityThread.main(ActivityThread.java:7656)
21:43:48.1616950Z 	at java.lang.reflect.Method.invoke(Native Method)
21:43:48.1617500Z 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
21:43:48.1618150Z 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)

Devices:

• Device: identical trio of API 30 emulators
• OS: Mac OS Big Sur 11.5.1

Let me know if I missed anything, or if there are further logs I can provide.

[Malinskiy]

@smugleafdev yeah, screenshots are accumulated into a GIF so that it’s not a 100 jpg files across. this was done to emulate the video capture behaviour on devices that do not support video recording for whatever reason. The gif will be empty though if there was no screenshot captured during the test. The 1 byte file is something new, I’ll investigate why this happens, but most likely the request for framebuffer: takes a very long time and the gif file is essentially empty. The empty ones are filtered out by marathon, but this 1 byte one is essentially empty as well.

It is unlikely that it’s adam bug: the screenshot functionality is E2E tested there on all APIs of devices as well as with stubs for different screen format versions.

If you use wireshark during the execution you can see how long it takes to capture screen from your device. I’ve seen real devices that might take upwards of 8 seconds. So if the test takes 5 seconds - the recording will be empty.

To investigate the issue with dying out devices I think it’s best to capture the traffic you have using something like tcpdump during the test run and analyze the output of the screenrecord there. Alternatively, you can place a breakpoint during your test run here to figure out the actual output.

I might also change the error handling there to include the actual output in the next version so that you see the output that cannot be parsed instead of relying on some debugging/network capture.

[Malinskiy]

I probably would try to look for some stacktraces in the logcat: sometimes adbd crashes or maybe some other internal component. By looking at the timestamp you might be able to find at least some logs related to this