[YouTube] Tab Extractor only extracts first page (30 videos) sometimes

Checklist

• I’m reporting a broken site support issue
• I’ve verified that I’m running youtube-dl version 2021.02.04.1
• I’ve checked that all provided URLs are alive and playable in a browser
• I’ve checked that all URLs and arguments with special characters are properly quoted or escaped
• I’ve searched the bugtracker for similar bug reports including closed ones
• I’ve read bugs section in FAQ

Verbose log

On this particular large channel sometimes it will only extract one page:

youtube-dl https://www.youtube.com/c/willaxtv/videos --flat-playlist --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.youtube.com/c/willaxtv/videos', '--flat-playlist', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.02.04.1
[debug] Python version 3.9.1 (CPython) - Linux-5.10.7-3-MANJARO-x86_64-with-glibc2.32
[debug] exe versions: ffmpeg 4.3.1, ffprobe 4.3.1, rtmpdump 2.4
[debug] Proxy map: {}
[youtube:tab] willaxtv: Downloading webpage
[download] Downloading playlist: Willax Television - Videos
[youtube:tab] playlist Willax Television - Videos: Downloading 30 videos
[download] Downloading video 1 of 30
[download] Downloading video 2 of 30
[download] Downloading video 3 of 30
[download] Downloading video 4 of 30
[download] Downloading video 5 of 30
[download] Downloading video 6 of 30
[download] Downloading video 7 of 30
[download] Downloading video 8 of 30
[download] Downloading video 9 of 30
[download] Downloading video 10 of 30
[download] Downloading video 11 of 30
[download] Downloading video 12 of 30
[download] Downloading video 13 of 30
[download] Downloading video 14 of 30
[download] Downloading video 15 of 30
[download] Downloading video 16 of 30
[download] Downloading video 17 of 30
[download] Downloading video 18 of 30
[download] Downloading video 19 of 30
[download] Downloading video 20 of 30
[download] Downloading video 21 of 30
[download] Downloading video 22 of 30
[download] Downloading video 23 of 30
[download] Downloading video 24 of 30
[download] Downloading video 25 of 30
[download] Downloading video 26 of 30
[download] Downloading video 27 of 30
[download] Downloading video 28 of 30
[download] Downloading video 29 of 30
[download] Downloading video 30 of 30
[download] Finished downloading playlist: Willax Television - Videos

Running again…

youtube-dl https://www.youtube.com/c/willaxtv/videos --flat-playlist --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.youtube.com/c/willaxtv/videos', '--flat-playlist', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.02.04.1
[debug] Python version 3.9.1 (CPython) - Linux-5.10.7-3-MANJARO-x86_64-with-glibc2.32
[debug] exe versions: ffmpeg 4.3.1, ffprobe 4.3.1, rtmpdump 2.4
[debug] Proxy map: {}
[youtube:tab] willaxtv: Downloading webpage
[download] Downloading playlist: Willax Television - Videos
[youtube:tab] Downloading page 1
[youtube:tab] Downloading page 2
[youtube:tab] Downloading page 3
[youtube:tab] Downloading page 4
[youtube:tab] Downloading page 5
[youtube:tab] Downloading page 6
[youtube:tab] Downloading page 7
..(and so on, this one would have thousands of pages. )

Description

When using --flat-playlist on a /videos tab it sometimes only downloads the first page.

Possibly similar to #27981. I accidentally wrote my debugging in there but realized this is probably different. Seems to be in these cases YouTube provides slightly different json data, causing the extractor to fail to process the next continuation page so we get less pages than there should be. My debugging of this: https://github.com/ytdl-org/youtube-dl/issues/27981#issuecomment-773239095

[pukkandan]

I have a fix: https://github.com/pukkandan/yt-dlp/commit/a1b535bd750ef33c3cd7cb95bbda3f93441788a5 But I’m tired of making PRs that never gets merged. Feel free to cherry pick it

[garbled1]

I can’t get the above patch to apply to 2021.2.4.1