SSA-in-MKV doesn't support simultaneous subtitles

ExoPlayer Version

2.17.1

Devices that reproduce the issue

• All devices I used. It’s not device-relevant.

• Devices I used: OnePlus 8T, OnePlus 10 Pro, Xiaomi Mi11 Ultra, Asus ROG Phone 4, Red Magic 6, Xiaomi Redmi Note 8 Pro, Samsung Galaxy S21

• All of the aforementioned devices run Android 10 to 12.

Devices that do not reproduce the issue

No response

Reproducible in the demo app?

Yes

Reproduction steps

• Loading any media file.
• Passing an SRT/ASS subtitle file to ExoPlayer that contains two (or more) lines that show up at the same time.

Expected result

• Both subtitles should appear at the same time.

Actual result

• If two subtitles are overlapping, the subtitle that appears later will remove the existence of the subtitle that appeared before it, albeit both co-existing during that timeframe.
• In many movies and series, subtitles overlap in some dialogues.

Media

OverlappingSubsDemo.zip

As for the video file, just grab any media from https://thedigitaltheater.com/dts-trailers/

Bug Report

• You will email the zip file produced by adb bugreport to [e-mail] after filing this issue.

[icbaker]

Thanks for clarifying that this specifically affects SSA inside MKV - that would explain why the commit I was referencing above isn’t ‘working’.

And thanks for providing example media, that will help to check any fix made for this works as expected.

The timestamps for SSA subtitles inside Matroska containers are extracted from the Matroska TimeStamp and BlockDuration elements. The MatroskaExtractor then emits a separate ExoPlayer ‘sample’ for each subtitle. This is in contrast to the way ‘sideloaded’ SSA subtitles are handled, where the whole file is emitted as a single ExoPlayer ‘sample’

The changes in https://github.com/google/ExoPlayer/pull/6595 assume that all the overlapping subtitles appear in the same ‘sample’.

We have some much larger subtitle changes in our future plans, which will change how subtitles map to ‘samples’, and should make the code paths between sideloaded SSA and SSA-in-MKV more similar. I’m afraid don’t think it’s worth trying to fix this issue with the current code, because it’s all going to change, and so the work would largely be wasted - but I will add it to the list of issues we should aim to resolve as part of the much larger refactor. Thanks for raising it 😃

[Reddnek]

ExampleFiles.zip

Regardless to MimeTypes such as ASS and SRT. There are two categories of subtitles I am talking about: Externally-loaded ones (manually loaded) and internal ones (coming in a MKV container).

The issue does NOT exist with manually-loaded SSA/ASS subtitle files. Here’s where the problem is:

• SRT subtitle tracks whether they’re manually loaded or internal in a MKV container.
• and ASS/SSA subtitle tracks that come in a MKV container (Internal subs). (only one sub shows up at a time).

I have attached the following files to reproduce the problem:

• 3 Subtitle Files: SSA, ASS and SRT - Produced by Aegisub and they all have identical subtitle lines.
• 2 Video files: 1 MP4 sample and 1 MKV sample that contains 2 subtitle tracks (One is SRT, the other is ASS).

Tested as of version 2.18.0. @icbaker @google-oss-bot I hope this helps.