9anime introduced... captcha

9anime has introduced google captcha. Thus I updated the script thinking perhaps this was resolved upstream, but not yet. So I am opening this bug.

To reproduce I have 9anime as default. On the other hand animepahe does work, twist.moe gives a link to vlc but is not working, and kissanime… well… I think cloudfare scraper needs an update. Anyway here is the command for 9anime:

▶ anime dl 'fruits basket' --play vlc
anime: anime-downloader 3.6.3
anime: 'NoneType' object has no attribute 'find_all'

LOG

▶ anime dl 'fruits basket' --play vlc  --log-level DEBUG
INFO root: anime-downloader 3.6.3
DEBUG root: Platform: Linux-5.0.10-050010-generic-x86_64-with-neon-18.04-bionic
DEBUG root: Python 3.6.7
DEBUG root: https://www4.9anime.to/search?keyword=fruits+basket
ERROR root: 'NoneType' object has no attribute 'find_all'

[GnatNamedAsh]

Didn’t want this getting too stale, so I decided to take a look at some of the code to see if it’s possible to fix what is happening with 9anime recently (once past the captcha). Most of the other providers I’ve tried either seem to have bad quality/intrusive watermarks or also started breaking recently.

The log of the error I was running into and fixed (also happens to be the error above too, but probably different traceback):

LOG

Traceback (most recent call last):
  File "/home/user/.local/bin//anime", line 11, in <module>
    sys.exit(main())
  File "/home/user/.local/lib/python3.6/site-packages/anime_downloader/cli.py", line 53, in main
    cli()
  File "/home/user/.local/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/user/.local/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/user/.local/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/user/.local/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/user/.local/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/user/.local/lib/python3.6/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/user/.local/lib/python3.6/site-packages/anime_downloader/commands/dl.py", line 93, in command
    fallback_qualities=fallback_qualities)
  File "/home/user/.local/lib/python3.6/site-packages/anime_downloader/sites/anime.py", line 84, in __init__
    self._episode_urls = self.get_data()
  File "/home/user/.local/lib/python3.6/site-packages/anime_downloader/sites/anime.py", line 165, in get_data
    self._episode_urls = self._scrape_episodes()
  File "/home/user/.local/lib/python3.6/site-packages/anime_downloader/sites/nineanime.py", line 125, in _scrape_episodes
    episodes = episodes.find_all('li')
AttributeError: 'NoneType' object has no attribute 'find_all'

I’ve been able to get the dl option for 9anime working for Dragon Ball Z. I’ve also been able to add it to the list of anime’s to watch. I mostly use the tool in Windows through WSL to download the episodes and then broadcast the file directory locally, so I can’t “prove” it watches correctly since there’s no actual player to load with WSL, so I’ll probably fork this repo, add the changes, then test the watch command using the Linux distro I have installed on another PC. I’ll also test a few other animes to see if they work properly before making the PR.

UDPATE (25/02/20): As an update, the day of going to test the watch functionality for 9anime (22/02/20), they imposed a pretty strict temporary IP ban/rate limiting rather recently that is affecting 3rd party apps as well as even normal users. Neither the dl or watch function will work and simply the search scrape triggers the rate limiting.

[gingerbeardman]

Just to say still not working.

$ anime -ll DEBUG dl yamada
2019-10-18 13:06:17 matt.local anime_downloader.util[18359] INFO anime-downloader 4.0.0
2019-10-18 13:06:17 matt.local anime_downloader.util[18359] DEBUG Platform: Darwin-18.7.0-x86_64-i386-64bit
2019-10-18 13:06:17 matt.local anime_downloader.util[18359] DEBUG Python 3.7.4
2019-10-18 13:06:18 matt.local anime_downloader.sites.helpers.request[18359] DEBUG HTML file temp_dir: /var/folders/sy/jcz9n9dn2t98677rzpmyvkr00000gn/T/animedll34cw94x
2019-10-18 13:06:18 matt.local anime_downloader.sites.helpers.request[18359] DEBUG -----
2019-10-18 13:06:18 matt.local anime_downloader.sites.helpers.request[18359] DEBUG GET https://www4.9anime.to/search?
2019-10-18 13:06:18 matt.local anime_downloader.sites.helpers.request[18359] DEBUG {'params': {'keyword': 'yamada'}}
2019-10-18 13:06:18 matt.local anime_downloader.sites.helpers.request[18359] DEBUG {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) Gecko/20100101 Firefox/56.0'}
2019-10-18 13:06:18 matt.local anime_downloader.sites.helpers.request[18359] DEBUG -----
send: b'GET /search?keyword=yamada HTTP/1.1\r\nHost: www4.9anime.to\r\nuser-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) Gecko/20100101 Firefox/56.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Fri, 18 Oct 2019 12:06:19 GMT
header: Content-Type: text/html
header: Transfer-Encoding: chunked
header: Connection: keep-alive
header: Set-Cookie: __cfduid=d826e6f22bb7787f37cb618148b80586f1571400379; expires=Sat, 17-Oct-20 12:06:19 GMT; path=/; domain=.9anime.to; HttpOnly
header: CF-Cache-Status: DYNAMIC
header: Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
header: Server: cloudflare
header: CF-RAY: 527a62b0d98edbdb-LHR
header: Content-Encoding: gzip
2019-10-18 13:06:19 matt.local anime_downloader.session[18359] DEBUG uncached request
2019-10-18 13:06:19 matt.local anime_downloader.sites.helpers.request[18359] DEBUG https://www4.9anime.to/search?keyword=yamada
2019-10-18 13:06:19 matt.local anime_downloader.sites.helpers.request[18359] DEBUG /var/folders/sy/jcz9n9dn2t98677rzpmyvkr00000gn/T/animedll34cw94x/tmp45u7uh0_
2019-10-18 13:06:19 matt.local anime_downloader.sites.nineanime[18359] DEBUG https://www4.9anime.to/search?keyword=yamada
Traceback (most recent call last):
  File "/usr/local/bin/anime", line 10, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/anime_downloader/cli.py", line 53, in main
    cli()
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/anime_downloader/commands/dl.py", line 85, in command
    anime_url = util.search(anime_url, provider)
  File "/usr/local/lib/python3.7/site-packages/anime_downloader/util.py", line 79, in search
    search_results = cls.search(query)
  File "/usr/local/lib/python3.7/site-packages/anime_downloader/sites/nineanime.py", line 84, in search
    'div', {'class': 'film-list'}).find_all('div', {'class': 'item'})
AttributeError: 'NoneType' object has no attribute 'find_all'
matt@matt:~$