Output stream delay of Twitch.tv streams

Checklist

• This is a bug report.
• This is a plugin request.
• This is a feature request.
• I used the search function to find already opened/closed issues or pull requests.

Description

Original feature request: streamlink/streamlink-twitch-gui#416

I’m not entirely sure if this is part of the HLS/m3u8 specs, but Twitch includes stream timestamps in their HLS playlists, so that a playback delay to the broadcaster can be calculated by their web player (see the created_at property of stream records from the Twitch API). I’m aware that there still can be an additional delay added by the local player when using Streamlink.

It would be useful for applications using Streamlink to read the current timestamp or even the calculated delay, if possible. This could be outputted as debug log message or a pipe (specified by a parameter) whenever a new playlist file (or stream chunk?) is being read.

If this data can’t be used, maybe all unrecognized m3u8 tags could be written as debug messages.

Example

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:5
#ID3-EQUIV-TDTG:2017-02-23T01:56:22
#EXT-X-MEDIA-SEQUENCE:21823
#EXT-X-TWITCH-ELAPSED-SYSTEM-SECS:87306.456
#EXT-X-TWITCH-ELAPSED-SECS:87299.595
#EXT-X-TWITCH-TOTAL-SECS:87330.456
#EXTINF:4.000,
index-0000021824-ZIka.ts
#EXTINF:4.000,
index-0000021825-7IPc.ts
#EXTINF:4.000,
index-0000021826-QQ9O.ts
#EXTINF:4.000,
index-0000021827-Outt.ts
#EXTINF:4.000,
index-0000021828-hT92.ts
#EXTINF:4.000,
index-0000021829-yrgH.ts

Stats of Twitch’s web player:

[bastimeyer]

It looks like this is a bit more complicated than I first thought. Well, at least what the Twitch web player is doing. It’s a bit of a pain debugging this, since the player code is quite large, is minified and contains compiled asm-js code.

From what it looks like, it is reading metadata of each mpeg-ts chunk and is then subtracting the offset time of the metadata from the current time + server time delta. I’ve reformatted and rewritten the part that sets the latency properties before it updates the DOM of the player stats menu:

When the SEGMENT_CHANGED event gets triggered, some parsing is being done (asm-js, incomprehensible) and the function below gets called with a metadata object as the first parameter:

function onID3Tag( data ) {
  var time = new Date().getTime() + this.serverTimeDelta;
  data.ID3.forEach( item => {
    if ( item.id === "TDEN" ) {
      this.latencyBroadcaster = time - new Date( item.info[0] ).getTime();
    } else if ( item.id === "TDTG" ) {
      this.latencyTranscoder = time - new Date( item.info[0] ).getTime();
    }
  });

I’m not sure what the “TDEN” and “TDTG” values mean and how they are extracted. I’ve tried reading the metadata of some mpeg-ts chunks with ffmpeg, but there’s nothing included other than the time offsets which are already included in the playlist (I think).

The server time delta is calculated from the EXT-X-TWITCH-INFO tag of the hls master playlist. Example:

#EXT-X-TWITCH-INFO:NODE="video-edge-c2aa38.fra02",MANIFEST-NODE-TYPE="weaver",MANIFEST-NODE="video-edge-c2aa38.fra02",SUPPRESS="true",SERVER-TIME="1487918069.54",USER-IP="xxx.xxx.xxx.xxx",SERVING-ID="a5c80608153e4fb39356a8976e2beee8",CLUSTER="fra02",ABS="false",BROADCAST-ID="24612292992",STREAM-TIME="18439.5420401",MANIFEST-CLUSTER="fra02"

serverTimeDelta = 1000 * Number( data["SERVER-TIME"] ) - new Date().getTime()

---

Is parsing the stream data really necessary here? The metadata included in the hls playlist doesn’t seem to be used by the web player, but I’m not entirely sure.

I’ve created a nodejs script that reads the created_at property of a stream record from the Twitch API and reads all twitch related tags from the most recent hls playlist when using the source stream quality. I don’t know what exactly the EXT-X-TWITCH-ELAPSED-SYSTEM-SECS, EXT-X-TWITCH-ELAPSED-SECS and EXT-X-TWITCH-TOTAL-SECS tags represent, but a calculation similar to what the web player is doing always yields ~30s, ~30s and ~4-8s (without using the server time delta, since I’m not reading the master playlist). Maybe the total-secs value could mean the offset time of the latest chunk. If this is the case, then it would be quite easy calculating the delay when the data of a specific chunk has been downloaded by streamlink and when it is being read by the video player.

[back-to]

to complicated for no real value