(Sucuri WAF) Unable to scrape https://footdistrict.com/en/
See original GitHub issuepip show cfscrape
output:
Name: cfscrape
Version: 2.0.7
Summary: A simple Python module to bypass Cloudflare's anti-bot page. See https://github.com/Anorov/cloudflare-scrape for more information.
Home-page: https://github.com/Anorov/cloudflare-scrape
Author: Anorov
Author-email: anorov.vorona@gmail.com
License: UNKNOWN
Location: /home/samy/.local/lib/python3.6/site-packages
Requires: requests
Required-by:
Code snippet experiencing the issue
#! /usr/bin/python2.7
import cfscrape, sys
from bs4 import BeautifulSoup
site = sys.argv[1]
scraper = cfscrape.create_scraper(delay=20)
# Or: scraper = cfscrape.CloudflareScraper() # CloudflareScraper inherits from requests.Session
page = scraper.get(site).content
soup = BeautifulSoup(page, "html.parser")
print(soup.encode('utf-8'))
Complete exception and traceback
No Exceptions.
URL of the Cloudflare-protected page
URL of Pastebin/Gist with HTML source of protected page
Issue Analytics
- State:
- Created 4 years ago
- Comments:5
Top Results From Across the Web
Sucuri Firewall Troubleshooting Guide
Sucuri Firewall Troubleshooting Guide · Step 1: Is the WAF enabled? · Step 2: Is your Sucuri SSL configured? · Step 3: Are...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
@pat6969 Sorry to rain on your parade but …
It’s not safe to use js2py to run untrusted code. If you don’t own the target domain, you shouldn’t use it. The idea of using js2py has been revisited time and time again. We’d like to use it but the author himself states that it is insecure. The author doesn’t really care about EcmaScript spec compliance. It’s slow, etc… Anybody telling you otherwise is uinformed.
As such, the following applies to any code using js2py in this context: https://nvd.nist.gov/vuln/detail/CVE-2017-7235
Feel free to use the branch that I created but like I said, I don’t plan on keeping it updated thus it should be considered insecure too.
You can use my repository for this… It will give you a sucuri wfa cookie that you can use with your requests, https://github.com/pat6969/Sucuri-Cloudproxy-Bypass @samy-mohsen