Can't play Kaltura multiplexed HLS content that looks like non-multiplexed content

Have you read the FAQ and checked for duplicate open issues? yes

What version of Shaka Player are you using? 3.0.8

Can you reproduce the issue with our latest release version? Yes

Can you reproduce the issue with the latest code from master? Yes

Are you using the demo app or your own custom app? The issue is reproducible with both

If custom app, can you reproduce the issue using our demo app? The issue is reproducible with both

What browser and OS are you using?

For embedded devices (smart TVs, etc.), what model and firmware version are you using? N/A

What are the manifest and license server URIs? sent you the URI on [e-mail] Email Subject: GitHub Issue 3149

What did you do? Play the stream using shaka player.

What did you expect to happen? The stream should have played

What actually happened? failed to get buffered range for audio DOMException: Failed to read the ‘buffered’ property from ‘SourceBuffer’: This SourceBuffer has been removed from the parent media source.

Logfile shaka-player-demo.appspot.com-1612887429970.log

Console Screen Shot

[joeyparrish]

@joeyparrish wrote:

What the player understands from the playlist is “these segments are video only” and “these are audio only”, and so when we set up the MediaSource and SourceBuffer objects (browser-provided API), we tell the browser to expect that, too.

However, it appears that Chrome rejects the actual content segments if they don’t match the configuration, but Firefox does not. This is a matter of the browser’s MediaSource implementation, and out of our control. Firefox seems to tolerate this mismatch, but Chrome does not.

@bansalkirti wrote:

I understand that is something that depends on the browser. However, this is also true that HLS.js, JW Player, and other players are able to play the stream on chrome.

I am not an expert in those projects, but I can guess that they are doing one of two things:

1. Demuxing/remuxing the content, so that the browser consumes video and audio separately
2. Correctly interpreting the multiplexed content as such, and telling the browser what to expect in advance

Option 1 would require architectural changes in Shaka Player. We just aren’t set up to do that, and it’s not in our plans at this time.

Option 2 would be ideal. We could ignore the audio-only streams, because we don’t support a mix of audio-only and audio+video tracks. If we knew that the audio+video streams were, in fact, files that contained both, we already have logic to do that.

The problem is that the playlist itself has literally no clues to tell us that this is the case. As far as I can tell, it looks identical to content in which audio and video streams are strictly separate.

It says CODECS="avc1.640028,mp4a.40.2", which could in general mean either “this contains both avc1.640028 video and mp4a.40.2 audio” or “this is avc1.640028 video, with an associated audio group containing mp4a.40.2 audio”. The existence of the AUDIO="aac" attribute is how we differentiate, and it leads us to the wrong conclusion. But there is honestly no other reasonable conclusion we could reach from the playlist alone.

Now, other players that deal with the container can discover at demux time that the stream contains both audio + video, and it can react then. But we don’t do that. The browser does that for us.

Native players generally deal with content at the container level. And web-based players can, but we choose not to duplicate in JavaScript work that the browser is meant to do in this environment. In fact, web-based players that choose to demux content for any reason must then remux it before feeding it to the browser. This is a waste of the client’s resources, in our opinion, even though it does prevent us from being capable of playing content such as this.

@mardu wrote:

I am having the exact same issue with Kaltura nginx-vod-module HLS playlists. As the setting vod_hls_force_unmuxed_segments is by default off all the video variants and segments are multiplexed. Everything works perfectly with only one audio track. Then the multiplexed variant is used and no separate audio variants are created.

It sounds to me as if the default should be changed. Maybe that’s something worth bringing up with Kaltura, @mardu, or with whomever is sending you those streams, @bansalkirti.

@mardu wrote:

I was wondering if it’s somehow possible to preprocess the master.m3u8 playlist before the player loads it?

Absolutely! You can use something called a “response filter” in Shaka Player to manipulate the content of the playlist before we parse it. There is an example of a response filter in one of our tutorials, though it’s for a completely different purpose. You can check out the “license wrapping” tutorial for details.

In essence, you can register a response filter, check for the request type shaka.net.NetworkingEngine.RequestType.MANIFEST, then extract the text of the playlist from response.data. You can then modify the playlist text, which can then be encoded back to Uint8Array and assigned back to response.data.

I hope this helps!

[ismena]

We don’t! My guess is, Firefox just handles this scenario better and is not as picky about label vs actual content. Let me tag @joeyparrish (who is way smarter than yours truly, but also much busier, so he may or may not reply :p) and see if he has a better guess.