[Vimeo] Showcase: unable to download JSON metadata: HTTP Error 404: Not Found

This is my first ever bug report, please be gentle.

Checklist

• I’m reporting a broken site support
• I’ve verified that I’m running youtube-dl version 2021.04.26
• I’ve checked that all provided URLs are alive and playable in a browser
• I’ve checked that all URLs and arguments with special characters are properly quoted or escaped
• I’ve searched the bugtracker for similar issues including closed ones

Verbose log

[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://vimeo.com/showcase/8113353/video/535007251', '--no-playlist', '--video-password', 'PRIVATE', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.04.26
[debug] Python version 3.9.4 (CPython) - Linux-5.10.32-1-MANJARO-x86_64-with-glibc2.33
[debug] exe versions: ffmpeg 4.3.2, ffprobe 4.3.2, rtmpdump 2.4
[debug] Proxy map: {}
[vimeo] 535007251: Downloading webpage
[vimeo] 535007251: Downloading JSON metadata
[vimeo] 535007251: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by <HTTPError 404: 'Not Found'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/lib/python3.9/site-packages/youtube_dl/extractor/common.py", line 634, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 2279, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.9/urllib/request.py", line 523, in open
    response = meth(req, response)
  File "/usr/lib/python3.9/urllib/request.py", line 632, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.9/urllib/request.py", line 561, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 641, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

Description

Youtube-dl seems unable to download this video from Vimeo directly (i.e. without taking a detour by referring to the playlist).

The exact same method worked not so long ago, with other videos from this exact playlist. Now, even the videos that I successfully downloaded before, fail with the above error message.

In Another thread with a similar issue, someone suggested an IP block from Vimeo, but I’ve tried this through several different VPN servers (with --call-home to verify that a different IP was indeed recorded), and I got exactly the same result each time.

However, if I use a roundabout way, referring to the playlist and specifying the video number in the playlist, it downloads fine, despite the same error showing up as a warning (I censored the playlist name and the filename):

[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://vimeo.com/showcase/8113353', '--playlist-items', '3', '--video-password', 'PRIVATE', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.04.26
[debug] Python version 3.9.4 (CPython) - Linux-5.10.32-1-MANJARO-x86_64-with-glibc2.33
[debug] exe versions: ffmpeg 4.3.2, ffprobe 4.3.2, rtmpdump 2.4
[debug] Proxy map: {}
[vimeo:album] 8113353: Downloading JSON metadata
[vimeo:album] 8113353: Downloading JSON metadata
[vimeo:album] 8113353: Verifying the password
[download] Downloading playlist: [playlist name]
[vimeo:album] 8113353: Downloading page 1
[vimeo:album] playlist [playlist name]: Downloading 1 videos
[download] Downloading video 1 of 1
[vimeo] 535004840: Downloading webpage
[vimeo] 535004840: Downloading JSON metadata
[vimeo] 535004840: Downloading JSON metadata
WARNING: Unable to download JSON metadata: HTTP Error 403: Forbidden
[vimeo] 535004840: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 535004840: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 535004840: Downloading fastly_skyfire m3u8 information
[vimeo] 535004840: Downloading fastly_skyfire m3u8 information
[vimeo] 535004840: Downloading akfire_interconnect_quic MPD information
[vimeo] 535004840: Downloading akfire_interconnect_quic MPD information
[vimeo] 535004840: Downloading fastly_skyfire MPD information
[vimeo] 535004840: Downloading fastly_skyfire MPD information
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on 'https://skyfire.vimeocdn.com/1619712136-0x7fb769059556b1c8631c10d515cdca222373ac97/b973b77a-7617-49fa-a459-bd5a03424fd3/sep/video/013d4266/playlist.m3u8?query_string_ranges=1'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 939
[download] Destination: [filename]

So, I am able to download the video, but I find it strange that only the roundabout way works.

I’m not a developer or anything, but perhaps it would be desirable (if possible) to have youtube-dl automatically invoke the roundabout method if the direct method fails? I.e.:

1. Youtube-dl tries to download the video but fails with a ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found;
2. It identifies the playlist that the video is in, and its number in the playlist;
3. It replaces youtube-dl [video URL] from the original command with youtube-dl [playlist URL] --playlist-items [number].

Identifying the playlist isn’t difficult in this particular case, because it’s literally the video URL minus the /video/535007251 part. However, I’m not sure if the 404 error doesn’t prevent youtube-dl from figuring out the video number in the playlist.

Again, I’m not a developer and I might be completely off the mark here, but I’m just trying my best to be helpful.

If I screwed up something in this report or if more information is required, please let me know!

[Vangelis66]

@whitedb ( @lakesdan ) :

The specific “American Dad!” episode you had tried to fetch, as derived from your posted log, i.e.

https://www.itv.com/hub/american-dad/2a4263a0280

produces the error below, when that URI is actually loaded in a browser:

Episode not available

We're sorry, the episode you're looking for is either coming soon or is no longer available.
Here are some episodes you can watch now.

Thus, the error returned by yt-dl,

ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by <HTTPError 404: 'Not Found'>);

is to be expected… 😉 In order to fetch AD! episodes actually available on ITVHub, you’d have to follow what @dirkf wrote; the official yt-dl binaries don’t contain PR#28955 (yet), so you’ll get the

ERROR: Unable to download JSON metadata: HTTP Error 500: Internal Server Error (caused by HTTPError());

yt-dl error instead 👎 ; to remedy that second error, please use the binaries kindly provided by sleaux-meaux 😺 …

[dirkf]

For ITV, see #28906.