VBR source freezes the first chunk after a video track changes

Hi,

I have an AndroidTV app and some channels feature two video layers with different bitrates and each of those layers have variable bitrate. These sources are DASH and the video tracks avc1.64001f. We have other sources that are HLS with the classical CBR and multiple quality layers and they play correctly, so at least we know we haven’t broken the standard behavior.

Exoplayer version: r2.10.2

Steps to reproduce:

• Start playing a channel from the lowest layer. The first chunk (8 seconds) plays nicely.
• The player tries to jump to the highest layer videoSizeChanged [17.23, 21575.54, window=0, period=0, 1280, 720].
• For the first chunk of that layer, the audio continues but the video freezes. No buffering, no error, just freezes.
• The video unfreezes after that second chunk is finished and the third one starts and it plays perfectly from then on unless there is another layer change during the playback.

I’m debugging onLoadStarted, onLoadError and onLoadCompleted callbacks and everything looks fine.

The following logs cover since the media initialization when I start playing a new channel to the third chunk when it unfreezes. I highlighted the moment the track changes and the video freezes:

2020-10-26 15:17:00.116 22831-22831: [onLoadStarted]
     url: [...].mp4
     mediaType: media initialization . trackFormat: Format(1, null, video/mp4, video/avc, avc1.64001f, 2000000, null, [960, 540, 25.0], [-1, -1])
2020-10-26 15:17:00.159 22831-22831: [onLoadCompleted]
     url: [...].mp4
     mediaType: media initialization . trackFormat: Format(1, null, video/mp4, video/avc, avc1.64001f, 2000000, null, [960, 540, 25.0], [-1, -1])
     elapsedRealTimeSeg: 1810955. loadDurationMs: 44. bytesLoaded: 998
2020-10-26 15:17:00.159 22831-22831: [onLoadStarted]
     url: [...].m4s
     mediaType: media . trackFormat: Format(1, null, video/mp4, video/avc, avc1.64001f, 2000000, null, [960, 540, 25.0], [-1, -1])
2020-10-26 15:17:00.687 22831-22831 [onLoadCompleted]
     url: [...].m4s
     mediaType: media . trackFormat: Format(1, null, video/mp4, video/avc, avc1.64001f, 2000000, null, [960, 540, 25.0], [-1, -1])
     elapsedRealTimeSeg: 1810955. loadDurationMs: 535. bytesLoaded: 778151
2020-10-26 15:17:00.687 22831-22831: onPlaybackStateChanged. playWhenReady: true. playbackState: 3
2020-10-26 15:17:00.688 22831-22831: [onLoadStarted]
     url: [...].m4s
     mediaType: media . trackFormat: Format(1, null, video/mp4, video/avc, avc1.64001f, 2000000, null, [960, 540, 25.0], [-1, -1])
2020-10-26 15:17:01.040 22831-22831: videoSizeChanged [0.75, 21575.67, window=0, period=0, 960, 540]
2020-10-26 15:17:01.115 22831-22831: [onLoadCompleted]
     url: [...].m4s
     mediaType: media . trackFormat: Format(1, null, video/mp4, video/avc, avc1.64001f, 2000000, null, [960, 540, 25.0], [-1, -1])
     elapsedRealTimeSeg: 1810956. loadDurationMs: 400. bytesLoaded: 1086287
2020-10-26 15:17:01.121 22831-22831: [onLoadStarted]
     url: [...].mp4
     mediaType: media initialization . trackFormat: Format(3, null, video/mp4, video/avc, avc1.64001f, 4000000, null, [1280, 720, 25.0], [-1, -1])
2020-10-26 15:17:01.194 22831-22831: [onLoadCompleted]
     url: [...].mp4
     mediaType: media initialization . trackFormat: Format(3, null, video/mp4, video/avc, avc1.64001f, 4000000, null, [1280, 720, 25.0], [-1, -1])
     elapsedRealTimeSeg: 1810956. loadDurationMs: 100. bytesLoaded: 996
2020-10-26 15:17:01.206 22831-22831: [onLoadStarted]
     url: [...].m4s
     mediaType: media . trackFormat: Format(3, null, video/mp4, video/avc, avc1.64001f, 4000000, null, [1280, 720, 25.0], [-1, -1])
2020-10-26 15:17:01.603 22831-22831: [onLoadCompleted]
     url: [...].m4s
     mediaType: media . trackFormat: Format(3, null, video/mp4, video/avc, avc1.64001f, 4000000, null, [1280, 720, 25.0], [-1, -1])
     elapsedRealTimeSeg: 1810956. loadDurationMs: 388. bytesLoaded: 670414
2020-10-26 15:17:08.063 22831-22831: [onLoadStarted]
     url: [...].m4s
     mediaType: media . trackFormat: Format(3, null, video/mp4, video/avc, avc1.64001f, 4000000, null, [1280, 720, 25.0], [-1, -1])
2020-10-26 15:17:08.209 22831-22831: [onLoadCompleted]
     url: [...].m4s
     mediaType: media . trackFormat: Format(3, null, video/mp4, video/avc, avc1.64001f, 4000000, null, [1280, 720, 25.0], [-1, -1])
     elapsedRealTimeSeg: 1810963. loadDurationMs: 137. bytesLoaded: 638433
2020-10-26 15:17:16.053 22831-22831: [onLoadStarted]
     url: [...].m4s
     mediaType: media . trackFormat: Format(3, null, video/mp4, video/avc, avc1.64001f, 4000000, null, [1280, 720, 25.0], [-1, -1])
2020-10-26 15:17:16.237 22831-22831: [onLoadCompleted]
     url: [...].m4s
     mediaType: media . trackFormat: Format(3, null, video/mp4, video/avc, avc1.64001f, 4000000, null, [1280, 720, 25.0], [-1, -1])
     elapsedRealTimeSeg: 1810971. loadDurationMs: 187. bytesLoaded: 2163852

**2020-10-26 15:17:17.035 22831-22831 D/EventLogger: videoSizeChanged [16.75, 21575.52, window=0, period=0, 1280, 720]**

2020-10-26 15:17:24.042 22831-22831: [onLoadStarted]
     url: [...].m4s
     mediaType: media . trackFormat: Format(3, null, video/mp4, video/avc, avc1.64001f, 4000000, null, [1280, 720, 25.0], [-1, -1])
2020-10-26 15:17:24.227 22831-22831: [onLoadCompleted]
     url: [...].m4s
     mediaType: media . trackFormat: Format(3, null, video/mp4, video/avc, avc1.64001f, 4000000, null, [1280, 720, 25.0], [-1, -1])
     elapsedRealTimeSeg: 1810979. loadDurationMs: 186. bytesLoaded: 2158411
2020-10-26 15:17:32.052 22831-22831: [onLoadStarted]
     url: [...].m4s
     mediaType: media . trackFormat: Format(3, null, video/mp4, video/avc, avc1.64001f, 4000000, null, [1280, 720, 25.0], [-1, -1])
2020-10-26 15:17:32.231 22831-22831: [onLoadCompleted]

Here is some other info I just took, again, I’m highlighting the moment when the track changes:

2020-10-28 09:43:09.097 26289-26289 D/EventLogger: decoderInitialized [0.38, 21575.67, window=0, period=0, video, OMX.amlogic.avc.decoder.awesome.secure]
2020-10-28 09:43:09.098 26289-26289 D/EventLogger: decoderInputFormatChanged [0.38, 21575.67, window=0, period=0, video, id=1, mimeType=video/avc, bitrate=2000000, codecs=avc1.64001f, res=960x540, fps=25.0]
2020-10-28 09:43:09.122 26289-26945 I/OMXClient: IOmx service obtained
2020-10-28 09:43:09.123 3150-1692/? I/OMXMaster: makeComponentInstance(OMX.google.aac.decoder) in omx@1.0-service process
2020-10-28 09:43:09.133 3150-1692/? E/OMXNodeInstance: setConfig(0xee41ccc0:google.aac.decoder, ConfigPriority(0x6f800002)) ERROR: Undefined(0x80001001)
2020-10-28 09:43:09.134 3150-1692/? E/OMXNodeInstance: setConfig(0xee41ccc0:google.aac.decoder, ConfigOperatingRate(0x6f800003)) ERROR: Undefined(0x80001001)
2020-10-28 09:43:09.136 3150-1692/? E/OMXNodeInstance: getConfig(0xee41ccc0:google.aac.decoder, ConfigAndroidVendorExtension(0x6f100004)) ERROR: Undefined(0x80001001)
2020-10-28 09:43:09.189 26289-26289 D/EventLogger: decoderInitialized [0.47, 21575.67, window=0, period=0, audio, OMX.google.aac.decoder]
2020-10-28 09:43:09.190 26289-26289 D/EventLogger: decoderInputFormatChanged [0.47, 21575.67, window=0, period=0, audio, id=0, mimeType=audio/mp4a-latm, bitrate=132184, codecs=mp4a.40.2, channels=2, sample_rate=48000, language=spa]
2020-10-28 09:43:09.291 3150-1692/? D/OmxVideoDecoder: getConfig OMX_IndexConfigCommonOutputCrop pRect->nWidth=960, pRect->nHeight=540
2020-10-28 09:43:09.301 3150-29256/? E/OMXNodeInstance: getConfig(0xee403ba8:amlogic.avc.decoder.awesome.secure, ??(0x7f00000c)) ERROR: UnsupportedSetting(0x80001019)
2020-10-28 09:43:09.518 26289-26289 D/EventLogger: videoSizeChanged [0.80, 21575.67, window=0, period=0, 960, 540]
2020-10-28 09:43:16.569 26289-26289 D/ExoPlayerPlaybackControl: [onTimelineChanged] buffer (ms): 25108  - player position: 21574561  - buffer position: 21599670  - buffer percentage: 99
2020-10-28 09:43:24.594 26289-26289 D/EventLogger: decoderInputFormatChanged [15.87, 21582.60, window=0, period=0, video, id=3, mimeType=video/avc, bitrate=4000000, codecs=avc1.64001f, res=1280x720, fps=25.0]
2020-10-28 09:43:24.662 26289-26289: [onTimelineChanged] buffer (ms): 25013  - player position: 21574657  - buffer position: 21599670  - buffer percentage: 99
2020-10-28 09:43:25.463 3150-1692/? D/OmxVideoDecoder: getConfig OMX_IndexConfigCommonOutputCrop pRect->nWidth=1280, pRect->nHeight=720
2020-10-28 09:43:25.465 3150-1692/? E/OMXNodeInstance: getConfig(0xee403ba8:amlogic.avc.decoder.awesome.secure, ??(0x7f00000c)) ERROR: UnsupportedSetting(0x80001019)

**2020-10-28 09:43:25.530 26289-26289 D/EventLogger: videoSizeChanged [16.81, 21575.53, window=0, period=0, 1280, 720]**

2020-10-28 09:43:32.507 26289-26289: [onTimelineChanged] buffer (ms): 25168  - player position: 21574501  - buffer position: 21599670  - buffer percentage: 99

This line, however: 2020-10-28 09:43:09.301 3150-29256/? E/OMXNodeInstance: getConfig(0xee403ba8:amlogic.avc.decoder.awesome.secure, ??(0x7f00000c)) ERROR: UnsupportedSetting(0x80001019) appears everythime the decoder initializes even with the channels that don’t have this problem, so I don’t think it’s related.

Thanks in advance, ask anything else you need to help me finding our bug. It’s the first time I work with VBR sources and I haven’t developed the application from the start so the knowledge I have about this is somewhat limited.

Link to a manifest with this properties: https://live-tcsgra.thechannel.store/testdashexoplayer/testdashexoplayer.mpd

[aiescola]

Thank you very much @ojw28. Bearing in mind that the profile is $Time and not $Number based, the startNumber parameter should not be necessary for the player. We assume that ExoPlayer is using it for any control apart from timing itself:

• Is there anyway to configure/eliminate such dependency?
• Can Exoplayer work with manifests $Time based without the startNumber attribute? Like those in Dashif examples of reference player: (be aware of redirection if tested)

https://livesim.dashif.org/livesim/segtimeline_1/testpic_2s/Manifest.mpd
https://wowzaec2demo.streamlock.net/live/bigbuckbunny/manifest_mvtime.mpd

It is true that the startNumber should be aligned, but the samples are actually aligned in the timeline needed for the requests of this profile. In the example, both representation share <S t="1098625650304" d="720000" r="2700"/>.

[ojw28]

I could not reproduce the problem with the provided manifest. Is this issue device specific, or are you seeing it across all devices? Please could you provide some more precise reproduction steps, ideally using the ExoPlayer demo app (with minimal modifications, if necessary).