Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Getting 403 on all requests. CF might have pushed an update.

See original GitHub issue

Before creating an issue, first upgrade cfscrape with pip install -U cfscrape and see if you’re still experiencing the problem. Please also confirm your Node version (node --version or nodejs --version) is version 10 or higher.

Make sure the website you’re having issues with is actually using anti-bot protection by Cloudflare and not a competitor like Imperva Incapsula or Sucuri. And if you’re using an anonymizing proxy, a VPN, or Tor, Cloudflare often flags those IPs and may block you or present you with a captcha as a result.

Please confirm the following statements and check the boxes before creating an issue:

  • I’ve upgraded cfscrape with pip install -U cfscrape
  • I’m using Node version 10 or higher
  • The site protection I’m having issues with is from Cloudflare
  • I’m not using Tor, a VPN, or an anonymizing proxy

Python version number

Run python --version and paste the output below:

Python 3.7.5

cfscrape version number

Run pip show cfscrape and paste the output below:

Name: cfscrape
Version: 2.1.1
Summary: A simple Python module to bypass Cloudflare's anti-bot page. See for more information.
Author: Anorov
License: UNKNOWN
Location: /usr/local/lib/python3.7/site-packages
Requires: requests

Code snippet involved with the issue

>>> import cfscrape
>>> scraper = cfscrape.CloudflareScraper()
>>> r = scraper.get("")
<Response [403]>

Complete exception and traceback

(If the problem doesn’t involve an exception being raised, leave this blank)

URL of the Cloudflare-protected page

URL of Pastebin/Gist with HTML source of protected page

Getting error 403 on almost every cf sites.

Issue Analytics

  • State:open
  • Created 4 years ago
  • Reactions:1
  • Comments:12

github_iconTop GitHub Comments

wsch-wacommented, Apr 3, 2020

i am getting the cookies but that cookies does not come from proxy instead it comes from my real IP. That is the issue the proxy is being ignored.

My use case to be super clear on the issue:

  • Browser (FF, IE,Chrome) shows the site without Captcha
  • I am not using a proxy
  • cfscrape returns a 403 error code which seems to be not representing the reality. The body-text shows “Normal content”.
  • Using Browsers I receive status 200 using F12 Debugging the traffic.
  • For me the headers of Browser and cfscrape look similar it is just the status 200 vs. 403

Since I have no clue what information I have to provide to fix the problem, I provide the Browsers sent request and received answer. ---- Browser Sent Request {“Anfragekopfzeilen (386 B)”:{“headers”:[{“name”:“Accept”,“value”:“text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8”},{“name”:“Accept-Encoding”,“value”:“gzip, deflate, br”},{“name”:“Accept-Language”,“value”:“de,en-US;q=0.7,en;q=0.3”},{“name”:“Connection”,“value”:“keep-alive”},{“name”:“DNT”,“value”:“1”},{“name”:“Host”,“value”:“”},{“name”:“Upgrade-Insecure-Requests”,“value”:“1”},{“name”:“User-Agent”,“value”:“Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:74.0) Gecko/20100101 Firefox/74.0”}]}}

---- Received Answer {“Antwortkopfzeilen (1,459 KB)”:{“headers”:[{“name”:“cache-control”,“value”:“no-cache”},{“name”:“cf-cache-status”,“value”:“DYNAMIC”},{“name”:“cf-ray”,“value”:“57e1b846faa6cba0-VIE”},{“name”:“content-encoding”,“value”:“gzip”},{“name”:“content-type”,“value”:“text/html; charset=utf-8”},{“name”:“date”,“value”:“Fri, 03 Apr 2020 09:21:31 GMT”},{“name”:“expect-ct”,“value”:“max-age=604800, report-uri="\”“},{“name”:“expires”,“value”:”-1"},{“name”:“pragma”,“value”:“no-cache”},{“name”:“server”,“value”:“cloudflare”},{“name”:“set-cookie”,“value”:“__cfduid=d0ba11b1bc5d1c04021503cd305a7ee481585905690; expires=Sun, 03-May-20 09:21:30 GMT; path=/;; HttpOnly; SameSite=Lax”},{“name”:“set-cookie”,“value”:“AWSALB=+yuMNwAu+kaIeFGsZoE/gEgxcqYLWviotTfkwIBRTZldvf1vW0mQ7l/hrNkE0jeAAJ1SsKTOwXEJXXaRM2gdeyUOt6YqMM4m83I3vDm3lShl+SyeaUXeNDcW9pqG; Expires=Fri, 10 Apr 2020 09:21:30 GMT; Path=/”},{“name”:“set-cookie”,“value”:“AWSALBCORS=+yuMNwAu+kaIeFGsZoE/gEgxcqYLWviotTfkwIBRTZldvf1vW0mQ7l/hrNkE0jeAAJ1SsKTOwXEJXXaRM2gdeyUOt6YqMM4m83I3vDm3lShl+SyeaUXeNDcW9pqG; Expires=Fri, 10 Apr 2020 09:21:30 GMT; Path=/; SameSite=None; Secure”},{“name”:“set-cookie”,“value”:“Unique_ID_v2=0dcb9c0455fc4d7589e75c024408615e;; expires=Wed, 03-Apr-2030 09:21:31 GMT; path=/”},{“name”:“set-cookie”,“value”:“__cf_bm=3cd18abfb0a930880bac2cc3a829276760cb4fba-1585905691-1800-ASOZlhSsbwiJ+ImNKM4F5d1gy9QkDueAfXcagsYDKar7m817Ju2aCCXZOKdVAISFWbyo4XQJshOFSWWsyGT2bFg=; path=/; expires=Fri, 03-Apr-20 09:51:31 GMT;; HttpOnly; Secure; SameSite=None”},{“name”:“strict-transport-security”,“value”:“max-age=15768000”},{“name”:“x-aspnet-version”,“value”:“4.0.30319”},{“name”:“x-aspnetmvc-version”,“value”:“5.2”},{“name”:“X-Firefox-Spdy”,“value”:“h2”},{“name”:“x-frame-options”,“value”:“SAMEORIGIN”},{“name”:“x-frame-options”,“value”:“SAMEORIGIN”},{“name”:“x-mvc-supplant-cachable”,“value”:“true”},{“name”:“x-ua-compatible”,“value”:“IE=edge,chrome=1”}]}}

restylercommented, Nov 23, 2021

My use case to be super clear on the issue:

  • Browser (FF, IE,Chrome) shows the site without Captcha
  • I am not using a proxy
  • cfscrape returns a 403 error code which seems to be not representing the reality.

I think this is probably not an “under attack” cloudflare protection but a tls fingerprint protection then. try this solution to confirm…

Read more comments on GitHub >

github_iconTop Results From Across the Web

403 Forbidden error: What it is and how to fix it
A 403 error is an HTTP status code that means “access denied.” If you ever had a treehouse as a kid, you may...
Read more >
What Is the 403 Forbidden Error and How to Fix It (8 ...
Often, HTTP 403 forbidden errors are caused by an access misconfiguration on the client-side, which means you can usually resolve the issue yourself....
Read more >
OSError: Tunnel connection failed: 403 Forbidden : Forums
I want to run a daily task that gets some small .json files and uses them to update a .txt file in my...
Read more >
Pushing to Git returning Error Code 403 fatal
Show activity on this post. A 403 code is "Forbidden". The server saw your request and refused it. Do you have permission to...
Read more >
Community Tip - Fixing Error 403 Forbidden
Try using a different browser, or use a private/incognito window. Your DNS cache may be pointing to the origin server. You're only seeing...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found