Embedded closed captions in DASH removed from text tracks on manifest refresh

Have you read the FAQ and checked for duplicate open issues?

Yes

What version of Shaka Player are you using?

3.0.3

Can you reproduce the issue with our latest release version?

Yes

Can you reproduce the issue with the latest code from master?

Yes

Are you using the demo app or your own custom app?

Demo

If custom app, can you reproduce the issue using our demo app?

N/A

What browser and OS are you using?

Chrome on MacOS

For embedded devices (smart TVs, etc.), what model and firmware version are you using?

N/A

What are the manifest and license server URIs?

Problem is not apparent (live manifest doesn’t refresh, at least not for 100 years): https://livesim.dashif.org/livesim/testpic_2s/cea608_and_segs.mpd

vs

Problem is apparent (live manifest refreshes every 5 seconds, problem apparent after first refresh) https://livesim.dashif.org/livesim/mup_5/testpic_2s/cea608_and_segs.mpd

What did you do?

For live manifests with embedded closed captions, Shaka successfully parses and presents the captions as a text track on first load of the manifest, but on subsequent loads the track is removed.

For instance, in the demo app using the second listed manifest above:

1. Open https://shaka-player-demo.appspot.com/demo/#audiolang=en-US;textlang=en-US;uilang=en-US;asset=https://livesim.dashif.org/livesim/mup_5/testpic_2s/cea608_and_segs.mpd;panel=CUSTOM CONTENT;build=uncompiled
2. Open the console and run shakaDemoMain.player_.getTextTracks()
3. After 5 seconds or more (after the manifest refreshes) run the above again

What did you expect to happen?

getTextTracks() returns all three text tracks: the subtitles track and the two embedded captions tracks (CC1 and CC3)

What actually happened?

On first load Shaka displays all three text tracks, but after the manifest refreshes, the two embedded captions tracks (CC1 and CC3) are no longer returned

Additional Notes

Looking into the code, the problem appears to be somewhat related to a TODO: https://github.com/google/shaka-player/blob/e7965fe21962833955a4f98c11e87f8931b721fd/lib/player.js#L1773-L1776

The text streams are created for the captions tracks on load. However, since the captions tracks are only created on first load, once the DASH manifest refreshes, the period combiner has no reference to the created track, and the text streams are overwritten. This seems to happen here: https://github.com/google/shaka-player/blob/e7965fe21962833955a4f98c11e87f8931b721fd/lib/dash/dash_parser.js#L441

I was looking into a possible solution, but wanted your advice and/or your help. I wasn’t sure where the best spot was to put the fix, having not worked in the project before. A couple of solutions I was considering were:

1. Move the createTextStreamsForClosedCaptions_ function into lib/util/periods.js and call it once the variants are created in combinePeriods: https://github.com/google/shaka-player/blob/e7965fe21962833955a4f98c11e87f8931b721fd/lib/util/periods.js#L211 . The issue I see there is that the tracks are recreated even if they are pre-existing, and the ID will change since it’s using an external stream ID. One potential solution might be to store a reference to the text streams before updating the manifest, try to match up on originalId, and keep the pre-existing tracks that match.

2. Have the manifest parser create the text streams from the captions tracks. Since the createTextStreamsForClosedCaptions_ function only seems to use the video portion of the variants, and the presence of the closedCaptions property, it may not be necessary to wait for the variants to be created before creating the text streams.

However, I was not sure if either of these were appropriate fixes.

If you don’t have the time to address the issue, let me know what your thoughts are and I can try to put a PR together. But I’d be grateful if you had a fix you wanted to implement yourself.

Please let me know if you need any more information and what you think.

Thank you!

[gesinger]

Thank you @theodab !

[theodab]

Yeah, this looks like it works. Sorry it took a bit to get back to you, I ran into a secondary bug during testing, I think.