Download from funk.net fails with HTTP Error 403: Forbidden / HTTP Error 400: Bad Request

Checklist

• I’m reporting a broken site support
• I’ve verified that I’m running youtube-dl version 2020.03.08
• I’ve checked that all provided URLs are alive and playable in a browser
• I’ve checked that all URLs and arguments with special characters are properly quoted or escaped
• I’ve searched the bugtracker for similar issues including closed ones

Verbose log

$ youtube-dl --verbose  https://www.funk.net/channel/doctor-who-1164/journeys-end-ov-515545/doctor-who-staffel-4-ov-2473

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'https://www.funk.net/channel/doctor-who-1164/journeys-end-ov-515545/doctor-who-staffel-4-ov-2473']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.03.08
[debug] Python version 2.7.12 (CPython) - Linux-4.15.0-88-generic-x86_64-with-LinuxMint-18.1-serena
[debug] exe versions: ffmpeg 2.8.15, ffprobe 2.8.15, rtmpdump 2.4
[debug] Proxy map: {}
[Funk] 515545: Downloading JSON metadata
[Nexx] 515545: Downloading JSON metadata
[Nexx] 515545: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
[Nexx] 515545: Downloading MPD manifest
WARNING: Failed to download MPD manifest: HTTP Error 403: Forbidden
[Nexx] 515545: Downloading ISM manifest
WARNING: Failed to download ISM manifest: HTTP Error 403: Forbidden
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'http://nx-d09.akamaized.net/06be6b49-c771-4f80-9ceb-a658ebc5e751/515545_src_1920x1080_6000.mp4'
ERROR: unable to download video data: HTTP Error 400: Bad Request
Traceback (most recent call last):
  File "/home/yee/Develop/youtube-dl/youtube-dl/youtube_dl/YoutubeDL.py", line 1926, in process_info
    success = dl(filename, info_dict)
  File "/home/yee/Develop/youtube-dl/youtube-dl/youtube_dl/YoutubeDL.py", line 1865, in dl
    return fd.download(name, info)
  File "/home/yee/Develop/youtube-dl/youtube-dl/youtube_dl/downloader/common.py", line 366, in download
    return self.real_download(filename, info_dict)
  File "/home/yee/Develop/youtube-dl/youtube-dl/youtube_dl/downloader/http.py", line 341, in real_download
    establish_connection()
  File "/home/yee/Develop/youtube-dl/youtube-dl/youtube_dl/downloader/http.py", line 109, in establish_connection
    ctx.data = self.ydl.urlopen(request)
  File "/home/yee/Develop/youtube-dl/youtube-dl/youtube_dl/YoutubeDL.py", line 2238, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 400: Bad Request

Description

The video https://www.funk.net/channel/doctor-who-1164/journeys-end-ov-515545/doctor-who-staffel-4-ov-2473 plays in a browser (Firefox) without problems, but youtube-dl fails to download m3u8 information, MPD & ISM manifests with HTTP Error 403: Forbidden.

About 3 weeks ago I didn’t experience any problems downloading videos from this site. Additionally, the problem only seems to occur when the location is nx-d09.akamaized.net like

[debug] Invoking downloader on u'http://nx-d09.akamaized.net/06be6b49-c771-4f80-9ceb-a658ebc5e751/515545_src_1920x1080_6000.mp4'
ERROR: unable to download video data: HTTP Error 400: Bad Request

Other videos, e.g. https://www.funk.net/channel/true-demon-12035/camping-im-wald-nimmt-fuerchterliches-ende-01-1616275/true-demon-alle-folgen-4971 from a different location funk-01.akamaized.net can still be downloaded:

$ youtube-dl --verbose https://www.funk.net/channel/true-demon-12035/camping-im-wald-nimmt-fuerchterliches-ende-01-1616275/true-demon-alle-folgen-4971

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'https://www.funk.net/channel/true-demon-12035/camping-im-wald-nimmt-fuerchterliches-ende-01-1616275/true-demon-alle-folgen-4971']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.03.08
[debug] Python version 2.7.12 (CPython) - Linux-4.15.0-88-generic-x86_64-with-LinuxMint-18.1-serena
[debug] exe versions: ffmpeg 2.8.15, ffprobe 2.8.15, rtmpdump 2.4
[debug] Proxy map: {}
[Funk] 1616275: Downloading JSON metadata
[Nexx] 1616275: Downloading JSON metadata
[Nexx] 1616275: Downloading m3u8 information
[Nexx] 1616275: Downloading MPD manifest
[Nexx] 1616275: Downloading ISM manifest
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'http://funk-01.akamaized.net/098fec4a-c9a8-446e-bcb0-d4ae19f2612c/1616275_src.ism/QualityLevels(5978504)/Manifest(video,format=m3u8-aapl)'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 120
[download] Destination: Camping im Wald nimmt fürchterliches Ende #01-1616275.fazure-hls-6257.mp4
[download]   2.9% of ~567.00MiB at  5.55MiB/s ETA 01:46^C
ERROR: Interrupted by user

Thank you for your work.

[lukasfink1]

@yeeeha Thank you for your work. It seems astonishing hard to at least just point out the line numbers where the presumed code duplication might be.

[yeeeha]

My pull request #24488 won’t be accepted. Sorry to say it, but trying to collaborate on this project really gives you a PITA… If someone’s interested in the fix: You may have a look at https://github.com/yeeeha/youtube-dl/tree/fix-24309 I’m out. Thank you and goodbye.