4anime download problem

Describe the bug

4anime recently moved servers as per their notice and because of this URLs returns 403.

To reproduce

I tried with Black clover’s episode 140. Downloader returns a 403 not found for the episode but When i tried to see if the episode is actually there or not on the website, it was there. And I was able to download the file.

anime --log-level DEBUG dl 'black clover' --provider 4anime --episodes 140

LOG

┌─[shen][wolf][~/Downloads/Video/Black-Clover]
└─▪ anime --log-level DEBUG dl 'black clover' --provider 4anime --episodes 140
2020-08-25 21:22:09 wolf anime_downloader.util[17291] INFO anime-downloader 4.6.3
2020-08-25 21:22:09 wolf anime_downloader.util[17291] DEBUG Platform: Linux-4.19.0-10-amd64-x86_64-with-MX-19.2-patito_feo
2020-08-25 21:22:09 wolf anime_downloader.util[17291] DEBUG Python 3.7.3
2020-08-25 21:22:09 wolf anime_downloader.sites.helpers.request[17291] DEBUG -----
2020-08-25 21:22:09 wolf anime_downloader.sites.helpers.request[17291] DEBUG POST https://4anime.to/wp-admin/admin-ajax.php
2020-08-25 21:22:09 wolf anime_downloader.sites.helpers.request[17291] DEBUG {'data': {'action': 'ajaxsearchlite_search', 'aslp': 'black clover', 'asid': 1, 'options': 'qtranslate_lang=0&set_intitle=None&customset%5B%5D=anime'}}
2020-08-25 21:22:09 wolf anime_downloader.sites.helpers.request[17291] DEBUG {'user-agent': 'Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.66 Safari/535.11'}
2020-08-25 21:22:09 wolf anime_downloader.sites.helpers.request[17291] DEBUG -----
send: b'POST /wp-admin/admin-ajax.php HTTP/1.1\r\nHost: 4anime.to\r\nuser-agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.66 Safari/535.11\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nContent-Length: 132\r\nContent-Type: application/x-www-form-urlencoded\r\n\r\n'
send: b'action=ajaxsearchlite_search&asid=1&aslp=black+clover&options=qtranslate_lang%3D0%26set_intitle%3DNone%26customset%255B%255D%3Danime'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Tue, 25 Aug 2020 15:52:10 GMT
header: Content-Type: text/html; charset=UTF-8
header: Transfer-Encoding: chunked
header: Connection: keep-alive
header: Set-Cookie: __cfduid=d583682efd468bba91a0302e02d5d6e761598370729; expires=Thu, 24-Sep-20 15:52:09 GMT; path=/; domain=.4anime.to; HttpOnly; SameSite=Lax; Secure
header: X-Robots-Tag: noindex
header: X-Content-Type-Options: nosniff
header: Expires: Wed, 11 Jan 1984 05:00:00 GMT
header: Cache-Control: no-cache, must-revalidate, max-age=0
header: X-Frame-Options: SAMEORIGIN
header: Referrer-Policy: strict-origin-when-cross-origin
header: Vary: Accept-Encoding
header: CF-Cache-Status: DYNAMIC
header: cf-request-id: 04c7eae7820000e25bde252200000001
header: Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
header: Server: cloudflare
header: CF-RAY: 5c867a859a13e25b-SIN
header: Content-Encoding: gzip
2020-08-25 21:22:10 wolf anime_downloader.session[17291] DEBUG uncached request
2020-08-25 21:22:10 wolf anime_downloader.sites.helpers.request[17291] DEBUG https://4anime.to/wp-admin/admin-ajax.php
2020-08-25 21:22:10 wolf anime_downloader.sites.helpers.request[17291] DEBUG /tmp/animedlm6nqrool/tmp90wnsfno
+--------+--------------+--------+
|      1 | Black Clover |        |
|--------+--------------+--------|
|   SlNo | Title        | Meta   |
+--------+--------------+--------+
Enter the anime no: [1]: 1
2020-08-25 21:22:11 wolf anime_downloader.util[17291] INFO Selected Black Clover
2020-08-25 21:22:11 wolf anime_downloader.sites.anime[17291] INFO Extracting episode info from page
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG -----
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG GET https://4anime.to/anime/black-clover
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG {}
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG {'user-agent': 'Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.66 Safari/535.11'}
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG -----
2020-08-25 21:22:11 wolf anime_downloader.session[17291] DEBUG cached request
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG https://4anime.to/anime/black-clover
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG /tmp/animedlm6nqrool/tmpr35m_rab
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG -----
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG GET https://4anime.to/anime/black-clover
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG {}
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG {'user-agent': 'Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.66 Safari/535.11'}
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG -----
2020-08-25 21:22:11 wolf anime_downloader.session[17291] DEBUG cached request
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG https://4anime.to/anime/black-clover
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG /tmp/animedlm6nqrool/tmplmunts2j
2020-08-25 21:22:11 wolf anime_downloader.sites.anime[17291] DEBUG EPISODE IDS: length: 140, ids: ['https://4anime.to/black-clover-episode-1/?id=4332', 'https://4anime.to/black-clover-episode-2/?id=4333', 'https://4anime.to/black-clover-episode-3/?id=4334', 'https://4anime.to/black-clover-episode-4/?id=4335', 'https://4anime.to/black-clover-episode-5/?id=4336', 'https://4anime.to/black-clover-episode-6/?id=4337', 'https://4anime.to/black-clover-episode-7/?id=4338', 'https://4anime.to/black-clover-episode-8/?id=4339', 'https://4anime.to/black-clover-episode-9/?id=4340', 'https://4anime.to/black-clover-episode-10/?id=4341', 'https://4anime.to/black-clover-episode-11/?id=4342', 'https://4anime.to/black-clover-episode-12/?id=4343', 'https://4anime.to/black-clover-episode-13/?id=4344', 'https://4anime.to/black-clover-episode-14/?id=4345', 'https://4anime.to/black-clover-episode-15/?id=4346', 'https://4anime.to/black-clover-episode-16/?id=4347', 'https://4anime.to/black-clover-episode-17/?id=4348', 'https://4anime.to/black-clover-episode-18/?id=4349', 'https://4anime.to/black-clover-episode-19/?id=4350', 'https://4anime.to/black-clover-episode-20/?id=4351', 'https://4anime.to/black-clover-episode-21/?id=4352', 'https://4anime.to/black-clover-episode-22/?id=4353', 'https://4anime.to/black-clover-episode-23/?id=4354', 'https://4anime.to/black-clover-episode-24/?id=4355', 'https://4anime.to/black-clover-episode-25/?id=4356', 'https://4anime.to/black-clover-episode-26/?id=4357', 'https://4anime.to/black-clover-episode-27/?id=4358', 'https://4anime.to/black-clover-episode-28/?id=4359', 'https://4anime.to/black-clover-episode-29/?id=4360', 'https://4anime.to/black-clover-episode-30/?id=4361', 'https://4anime.to/black-clover-episode-31/?id=4362', 'https://4anime.to/black-clover-episode-32/?id=4363', 'https://4anime.to/black-clover-episode-33/?id=4364', 'https://4anime.to/black-clover-episode-34/?id=4365', 'https://4anime.to/black-clover-episode-35/?id=4366', 'https://4anime.to/black-clover-episode-36/?id=4367', 'https://4anime.to/black-clover-episode-37/?id=4368', 'https://4anime.to/black-clover-episode-38/?id=4369', 'https://4anime.to/black-clover-episode-39/?id=4370', 'https://4anime.to/black-clover-episode-40/?id=4371', 'https://4anime.to/black-clover-episode-41/?id=4372', 'https://4anime.to/black-clover-episode-42/?id=4373', 'https://4anime.to/black-clover-episode-43/?id=4374', 'https://4anime.to/black-clover-episode-44/?id=4375', 'https://4anime.to/black-clover-episode-45/?id=4377', 'https://4anime.to/black-clover-episode-46/?id=5504', 'https://4anime.to/black-clover-episode-47/?id=5965', 'https://4anime.to/black-clover-episode-48/?id=6731', 'https://4anime.to/black-clover-episode-49/?id=6957', 'https://4anime.to/black-clover-episode-50/?id=7271', 'https://4anime.to/black-clover-episode-51/?id=7955', 'https://4anime.to/black-clover-episode-52/?id=8518', 'https://4anime.to/black-clover-episode-53/?id=9111', 'https://4anime.to/black-clover-episode-54/?id=9527', 'https://4anime.to/black-clover-episode-55/?id=9882', 'https://4anime.to/black-clover-episode-56/?id=10161', 'https://4anime.to/black-clover-episode-57/?id=10508', 'https://4anime.to/black-clover-episode-58/?id=10629', 'https://4anime.to/black-clover-episode-59/?id=10820', 'https://4anime.to/black-clover-episode-60/?id=10879', 'https://4anime.to/black-clover-episode-61/?id=11179', 'https://4anime.to/black-clover-episode-62/?id=11242', 'https://4anime.to/black-clover-episode-63/?id=11305', 'https://4anime.to/black-clover-episode-64/?id=11509', 'https://4anime.to/black-clover-episode-65/?id=12048', 'https://4anime.to/black-clover-episode-66/?id=12303', 'https://4anime.to/black-clover-episode-67/?id=12366', 'https://4anime.to/black-clover-episode-68/?id=12458', 'https://4anime.to/black-clover-episode-69/?id=12536', 'https://4anime.to/black-clover-episode-70/?id=12630', 'https://4anime.to/black-clover-episode-71/?id=12748', 'https://4anime.to/black-clover-episode-72/?id=13477', 'https://4anime.to/black-clover-episode-73/?id=14383', 'https://4anime.to/black-clover-episode-74/?id=14706', 'https://4anime.to/black-clover-episode-75/?id=15246', 'https://4anime.to/black-clover-episode-76/?id=15778', 'https://4anime.to/black-clover-episode-77/?id=16246', 'https://4anime.to/black-clover-episode-78/?id=17160', 'https://4anime.to/black-clover-episode-79/?id=18058', 'https://4anime.to/black-clover-episode-80/?id=18677', 'https://4anime.to/black-clover-episode-81/?id=19421', 'https://4anime.to/black-clover-episode-82/?id=20085', 'https://4anime.to/black-clover-episode-83/?id=20634', 'https://4anime.to/black-clover-episode-84/?id=21140', 'https://4anime.to/black-clover-episode-85/?id=21412', 'https://4anime.to/black-clover-episode-86/?id=21639', 'https://4anime.to/black-clover-episode-87/?id=21933', 'https://4anime.to/black-clover-episode-88/?id=22391', 'https://4anime.to/black-clover-episode-89/?id=23156', 'https://4anime.to/black-clover-episode-90/?id=23257', 'https://4anime.to/black-clover-episode-91/?id=23620', 'https://4anime.to/black-clover-episode-92/?id=24300', 'https://4anime.to/black-clover-episode-93/?id=24718', 'https://4anime.to/black-clover-episode-94/?id=25128', 'https://4anime.to/black-clover-episode-95/?id=25176', 'https://4anime.to/black-clover-episode-96/?id=25606', 'https://4anime.to/black-clover-episode-97/?id=26127', 'https://4anime.to/black-clover-episode-98/?id=26688', 'https://4anime.to/black-clover-episode-99/?id=27361', 'https://4anime.to/black-clover-episode-100/?id=27819', 'https://4anime.to/black-clover-episode-101/?id=28222', 'https://4anime.to/black-clover-episode-102/?id=28680', 'https://4anime.to/black-clover-episode-103/?id=28709', 'https://4anime.to/black-clover-episode-104/?id=28815', 'https://4anime.to/black-clover-episode-105/?id=28922', 'https://4anime.to/black-clover-episode-106/?id=29016', 'https://4anime.to/black-clover-episode-107/?id=29153', 'https://4anime.to/black-clover-episode-108/?id=29303', 'https://4anime.to/black-clover-episode-109/?id=29408', 'https://4anime.to/black-clover-episode-110/?id=29468', 'https://4anime.to/black-clover-episode-111/?id=29529', 'https://4anime.to/black-clover-episode-112/?id=29599', 'https://4anime.to/black-clover-episode-113/?id=29660', 'https://4anime.to/black-clover-episode-114/?id=29740', 'https://4anime.to/black-clover-episode-115/?id=29806', 'https://4anime.to/black-clover-episode-116/?id=30315', 'https://4anime.to/black-clover-episode-117/?id=30658', 'https://4anime.to/black-clover-episode-118/?id=30861', 'https://4anime.to/black-clover-episode-119/?id=30995', 'https://4anime.to/black-clover-episode-120/?id=31162', 'https://4anime.to/black-clover-episode-121/?id=31230', 'https://4anime.to/black-clover-episode-122/?id=31583', 'https://4anime.to/black-clover-episode-123/?id=31729', 'https://4anime.to/black-clover-episode-124/?id=31776', 'https://4anime.to/black-clover-episode-125/?id=31829', 'https://4anime.to/black-clover-episode-126/?id=31886', 'https://4anime.to/black-clover-episode-127/?id=32290', 'https://4anime.to/black-clover-episode-128/?id=33238', 'https://4anime.to/black-clover-episode-129/?id=33370', 'https://4anime.to/black-clover-episode-130/?id=33467', 'https://4anime.to/black-clover-episode-131/?id=33510', 'https://4anime.to/black-clover-episode-132/?id=33543', 'https://4anime.to/black-clover-episode-133/?id=42280', 'https://4anime.to/black-clover-episode-134/?id=42550', 'https://4anime.to/black-clover-episode-135/?id=42718', 'https://4anime.to/black-clover-episode-136/?id=42825', 'https://4anime.to/black-clover-episode-137/?id=43009', 'https://4anime.to/black-clover-episode-138/?id=43113', 'https://4anime.to/black-clover-episode-139/?id=43150', 'https://4anime.to/black-clover-episode-140/?id=43189']
2020-08-25 21:22:11 wolf anime_downloader.commands.dl[17291] INFO Found anime: Black Clover
2020-08-25 21:22:11 wolf anime_downloader.sites.anime[17291] DEBUG Extracting stream info of id: https://4anime.to/black-clover-episode-140/?id=43189
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG -----
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG GET https://4anime.to/black-clover-episode-140/?id=43189
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG {}
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG {'user-agent': 'Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.66 Safari/535.11'}
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG -----
2020-08-25 21:22:11 wolf anime_downloader.session[17291] DEBUG cached request
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG https://4anime.to/black-clover-episode-140?id=43189
2020-08-25 21:22:11 wolf anime_downloader.sites.helpers.request[17291] DEBUG /tmp/animedlm6nqrool/tmpyu_pis2_
2020-08-25 21:22:12 wolf anime_downloader.sites.anime[17291] DEBUG Sources : [('no_extractor', 'https://v2.4animu.me/Black-Clover/Black-Clover-Episode-140-1080p.mp4?md5=BV2aiSxEAD0cZB4RG_qjaA&expires=1598380676')]
2020-08-25 21:22:12 wolf anime_downloader.commands.dl[17291] INFO Downloading to /home/shen/Downloads/Video/Black-Clover
2020-08-25 21:22:12 wolf anime_downloader.util[17291] DEBUG cmd: {aria2}
DEBUG:anime_downloader.util:cmd: {aria2}
2020-08-25 21:22:12 wolf anime_downloader.util[17291] DEBUG episode: <anime_downloader.sites._4anime.Anime4Episode object at 0x7fb5d2e9cc18>
DEBUG:anime_downloader.util:episode: <anime_downloader.sites._4anime.Anime4Episode object at 0x7fb5d2e9cc18>
2020-08-25 21:22:12 wolf anime_downloader.util[17291] DEBUG file format: {anime_title}/{anime_title}_{ep_no}
DEBUG:anime_downloader.util:file format: {anime_title}/{anime_title}_{ep_no}
2020-08-25 21:22:12 wolf anime_downloader.util[17291] DEBUG formatted cmd: aria2c https://v2.4animu.me/Black-Clover/Black-Clover-Episode-140-1080p.mp4?md5=BV2aiSxEAD0cZB4RG_qjaA&expires=1598380676 -x 12 -s 12 -j 12 -k 10M -o Black_Clover/Black_Clover_140.mp4 --continue=true --dir=/home/shen/Downloads/Video/Black-Clover --stream-piece-selector=inorder --min-split-size=5M --referer=https://v2.4animu.me/Black-Clover/Black-Clover-Episode-140-1080p.mp4?md5=BV2aiSxEAD0cZB4RG_qjaA&expires=1598380676 --check-certificate=false --user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36 --max-overall-download-limit=0
DEBUG:anime_downloader.util:formatted cmd: aria2c https://v2.4animu.me/Black-Clover/Black-Clover-Episode-140-1080p.mp4?md5=BV2aiSxEAD0cZB4RG_qjaA&expires=1598380676 -x 12 -s 12 -j 12 -k 10M -o Black_Clover/Black_Clover_140.mp4 --continue=true --dir=/home/shen/Downloads/Video/Black-Clover --stream-piece-selector=inorder --min-split-size=5M --referer=https://v2.4animu.me/Black-Clover/Black-Clover-Episode-140-1080p.mp4?md5=BV2aiSxEAD0cZB4RG_qjaA&expires=1598380676 --check-certificate=false --user-agent=Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36 --max-overall-download-limit=0

08/25 21:22:12 [NOTICE] Downloading 1 item(s)
[#b5586f 0B/0B CN:1 DL:0B]                                                                                                                           
08/25 21:22:13 [ERROR] CUID#7 - Download aborted. URI=https://v2.4animu.me/Black-Clover/Black-Clover-Episode-140-1080p.mp4?md5=BV2aiSxEAD0cZB4RG_qjaA&expires=1598380676
Exception: [AbstractCommand.cc:351] errorCode=22 URI=https://v2.4animu.me/Black-Clover/Black-Clover-Episode-140-1080p.mp4?md5=BV2aiSxEAD0cZB4RG_qjaA&expires=1598380676
  -> [HttpSkipResponseCommand.cc:240] errorCode=22 The response status is not successful. status=403

08/25 21:22:13 [NOTICE] Download GID#b5586f2a7e9d3e77 not complete: /home/shen/Downloads/Video/Black-Clover/Black_Clover/Black_Clover_140.mp4

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
b5586f|ERR |       0B/s|/home/shen/Downloads/Video/Black-Clover/Black_Clover/Black_Clover_140.mp4

Status Legend:
(ERR):error occurred.

aria2 will resume download if the transfer is restarted.
If there are any errors, then see the log file. See '-l' option in help/man page for details.

[Blatzar]

Sucessfully cracked 4anime using the internal downloader, but cannot replicate with aria2. Could there be some sort of aria2 detection?

[Blatzar]

I cannot replicate the browser request using aria2c, always getting 403 error. No idea how to solve.