Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

(Sucuri WAF) Unable to scrape https://footdistrict.com/en/

See original GitHub issue

pip show cfscrape output:

Name: cfscrape
Version: 2.0.7
Summary: A simple Python module to bypass Cloudflare's anti-bot page. See https://github.com/Anorov/cloudflare-scrape for more information.
Home-page: https://github.com/Anorov/cloudflare-scrape
Author: Anorov
Author-email: anorov.vorona@gmail.com
License: UNKNOWN
Location: /home/samy/.local/lib/python3.6/site-packages
Requires: requests
Required-by:

Code snippet experiencing the issue

#! /usr/bin/python2.7

import cfscrape, sys
from bs4 import BeautifulSoup
site = sys.argv[1]

scraper = cfscrape.create_scraper(delay=20)
# Or: scraper = cfscrape.CloudflareScraper()  # CloudflareScraper inherits from requests.Session
page = scraper.get(site).content
soup = BeautifulSoup(page, "html.parser")
print(soup.encode('utf-8'))

Complete exception and traceback

No Exceptions.

URL of the Cloudflare-protected page

https://footdistrict.com/en/

URL of Pastebin/Gist with HTML source of protected page

https://pastebin.com/raw/k6MSbDst

Issue Analytics

State:
Created 4 years ago
Comments:5

Top GitHub Comments

2reactions

ghostcommented, Sep 3, 2019

@pat6969 Sorry to rain on your parade but …

It’s not safe to use js2py to run untrusted code. If you don’t own the target domain, you shouldn’t use it. The idea of using js2py has been revisited time and time again. We’d like to use it but the author himself states that it is insecure. The author doesn’t really care about EcmaScript spec compliance. It’s slow, etc… Anybody telling you otherwise is uinformed.

As such, the following applies to any code using js2py in this context: https://nvd.nist.gov/vuln/detail/CVE-2017-7235

Feel free to use the branch that I created but like I said, I don’t plan on keeping it updated thus it should be considered insecure too.

0reactions

2020iscancelledcommented, Sep 3, 2019

You can use my repository for this… It will give you a sucuri wfa cookie that you can use with your requests, https://github.com/pat6969/Sucuri-Cloudproxy-Bypass @samy-mohsen