[Site Request] [Twitch] Dealing with pre-roll and mid-roll ads

Checklist

• I’m reporting a site feature request
• I’ve verified that I’m running yt-dlp version 2021.09.25
• I’ve searched the bugtracker for similar site feature requests including closed ones

Description

Twitch has a system that embeds ad segments directly into an M3U playlist. It doesn’t contain actual ad streams but you get placeholder segments lasting as long as ad would. Pre-roll ads play right after you start grabbing a live stream. They usually take about 15 seconds to finish. Mid-roll ads can take up to a few minutes. Both prevent you from getting actual stream content. This is how it looks.

Twitch does this only to live streams and reruns. There are no embedded ads if you download VODs (at least for now).

How other apps solve this

There are well known tools that successfully deal with Twitch ads. My understanding is it involves stripping-filtering-skipping ad segments from an M3U based on their # comments. At worst you get a few seconds gaps in a stream but you don’t get full length ads either.

Here is how streamlink feature –twitch-disable-ads works. https://github.com/streamlink/streamlink/blob/master/src/streamlink/plugins/twitch.py#L39

        is_ad = (
            daterange.classname == "twitch-stitched-ad"
            or str(daterange.id or "").startswith("stitched-ad-")
            or any(attr_key.startswith("X-TV-TWITCH-AD-") for attr_key in daterange.x.keys())
        )

There is also a filter list for uBlock Origin for web based player .

twitch.tv##+js(json-prune, /^X-TV-TWITCH-AD/)
...

At some point there were server side ways to prevent Twitch from serving you ads at all by cloaking as specific ad-free client. I’m not sure if any of them work right now. Related issues are https://github.com/streamlink/streamlink/issues/3210, https://github.com/streamlink/streamlink/issues/2894

Yt-dlp side of things

As far as I understand native HLS downloader doesn’t support Twitch at all because yt-dlp always^† switches to ffmpeg. Obviously native HLS downloader can be fixed and adapted to filter ads but it would cover just that i.e. stream downloading directly with yt-dlp.

I’m equally interested in ad filtering when external tools use yt-dlp just to extract an M3U URL with --get-url or --dump-json . That means they don’t consume stdin and yt-dlp would have to act as a local M3U proxy/webserver. Besides native HLS downloader has (had?) a lot of i/o issues and overall ffmpeg is preferred.

† There is a bug that prevents yt-dlp from switching to ffmpeg if you try to download rerun streams. Yt-dlp downloads small ad segment and then just gracefully exists as if stream ended.

Reproducing the issue

Pre-roll ads

1. Go to https://www.twitch.tv/directory/all
2. Pick any popular live streamer
3. yt-dlp https://www.twitch.tv/streamer
4. Wait about 20 seconds & cancel downloading
5. Verify that downloaded video starts with an ad segment

Alternatively you can just feed the result of yt-dlp -g https://www.twitch.tv/streamer to your favorite player that supports M3U or pipe yt-dlp -o - https://www.twitch.tv/streamer into it.

Mid-roll ads

Same as above but you would have to wait up to half an hour for ads to appear.

Reruns

† There is no simple way to find just reruns. You can either scroll through the whole Twitch directory or try to find a channel through this page - https://www.twitch.tv/search?term=rerun&type=channels

That’s how it looks on Twitch, rerun vs live -

Things to consider

Twitch is very inconsistent. Whether you get ads or not and the way you get them depends on many things like IP origin, a channel you pick, a number of advertisers in your regional ad pool, time of day, etc.

Everything above is just my own understanding of current state of things so there might be inaccuracies.

[reallyuniquename]

At some point there were server side ways to prevent Twitch from serving you ads at all by cloaking as specific ad-free client

https://github.com/pixeltris/TwitchAdSolutions/blob/master/bypass/bypass-ublock-origin.js There is one now.

Workaround is --add-header "X-Device-Id: twitch-web-wall-mason" --add-header "Device-ID: twitch-web-wall-mason".

[pukkandan]

If you are talking about livestream, read https://github.com/yt-dlp/yt-dlp/issues/1089#issuecomment-927302173. If you mean VODs, open a new issue