60 FPS Widevine DASH content stutters on some devices

Content description

60 FPS Widevine encrypted DASH streams stutter and/or have many dropped frames. The (per email) provided content shows a live ticker at the bottom, where this stuttering and frame drops can be observed very well.

There are players like the NexPlayer (https://www.nexplayersdk.com/), which shows better performance on the same content and devices, therefore we rule out a weak DRM decryption module or hardware decoders.

We also tried to find the bottleneck in the code base, tried to tweak several parameters, and implemented a decoder version using the asynchronous callback Android provides. Unfortunately without major improvements.

Additionally, on the more powerful/recent devices we observed that frames are provided to the Surface in time (ExoPlayer business logic did not drop the frames) but the stuttering was still visible. When we changed the code so that the renderTimeStamp passed to releaseOutputBuffer is not “reasonably close” to the current system time (which causes the Surface to ignore the timestamp and display the buffer at the earliest feasible time. In this mode the Surface will not drop frames), we did not see any stuttering/frame drops anymore as expected. However, over time video and audio were drifting apart as if video would not be playing at 60 FPS anymore but rather at a lower frame rate. This observation lead us to the assumption that the surface itself maybe cannot keep up with the high frame rate.

Link to test content

Provided per email.

Version of ExoPlayer being used

2.10.4 and at least back to 2.9.6

Device(s) and version(s) of Android being used

Seems unrelated to the Android version. Seen issues with:

• OnePlus 5T
• OnePlus 5
• Galaxy Tab E (SM-T337W) | 7.1.1, 6.0.1
• Galaxy Tab 3 Lite (SM-T113) | 4.4.4
• Galaxy S8 (Canadian and “regular” version) | 9.0.0
• OnePlus 5 | 9.0.0
• Google Pixel 3 | 9.0.0
• Samsung S6 (SM-G928V) | 7.0.0
• Samsung S7 (SM-G930T1) | 8.0.0
• Galaxy Tab 3 10.1 (GT-P5210)
• Huawei P30 Lite (MAR-LX3A)
• Galaxy Tab 3 (SM-T310)
• Galaxy Tab S3 (SM-T820)
• Galaxy Tab S2 (SM-T810) | 6.0.1,7.0,7.0
• Galaxy Core (SM-G386) | 4.4.2
• Galaxy Tab 4 (SM-T230) | 4.4.2
• MiBox3 (MIBOX3) | 8.0.0
• Galaxy A8 (SM-A530W) | 7.1.1,8.0.0,9

A full bug report captured from the device

Provided per email.

[christosts]

We pushed commit 7a2c7c3 that promotes the experimental asynchronous queueing feature to be used by default. This will be included on the next release (2.16.0) which is expected to be out during November 2021.

The player will use this feature by default on devices with Android 12+, but it will not enable it automatically for devices running previous Android versions. Asynchronous queueing largely addressed stuttering and dropped frames observed on some devices when playing high frame Widevine-encrypted content. However, we observed some devices can be fragile to the additional threads introduced, especially on older platforms (e.g., prior to Android 8). The number of issues was very low, yet is prevented us from widely enabling the feature on all platforms.

The commit adds APIs to enable/disable the feature, therefore you can manually enable it on a device running an earlier Android version.

I will close this issue. If you observe any issues with the new feature, please file a new ticket in order to focus the conversation on the specific feature. Thanks.

[christosts]

@ojw28 any update on this?

We plan to include this API as experimental (therefore subject to change) in 2.12. We cannot promise a date yet though.