Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Getting "just a moment"

See original GitHub issue

I can’t make it work. I’m just getting redirect, this is my code:

scraper = cfscrape.create_scraper()  # returns a CloudflareScraper instance
    headers = {'referer': 'http://seriesblanco.com/'}
    just_loggin = scraper.get(form.getvalue('url')) 
    print scraper.get(form.getvalue('url') , headers=headers).content  # => "<!DOCTYPE html><html><head>..."

Is something wrong with it?, thanks in advance.

Getting the page:

Checking your browser before accessing seriesblanco.com.

This process is automatic. Your browser will redirect to your requested content shortly. Please allow up to 5 seconds… DDoS protection by CloudFlare Ray ID: 2e04c766319425b5

And then:

The requested URL /cdn-cgi/l/chk_jschl was not found on this server. on my server.

Issue Analytics

State:
Created 7 years ago
Comments:10

Top GitHub Comments

1reaction

platedodevcommented, Sep 13, 2016

@Twoure I did a git clone of cloudflare-scrape master and now works, thanks mate 😃.

1reaction

Twourecommented, Sep 11, 2016

@platedodev Looks like you tried to combine Example 1 & 3, but left out some parts. Assuming you wanted Example 3, the code should look as follows:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-

# enable debugging
import cgi
import cgitb
import cfscrape
import requests

USER_AGENT = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36"

cgitb.enable()
form = cgi.FieldStorage()

print "Content-type: text/html"
print ""
if not form.getvalue('url'):
    print "No url."
else:
    url = form.getvalue('url');
    base_url = "http://" if url.startswith("http:") else "https://" + url.rsplit('/')[2]
    http_headers = {'User-Agent': USER_AGENT}
    tokens = cfscrape.get_cookie_string(url=base_url, user_agent=USER_AGENT)
    http_headers.update({'Cookie': tokens[0], 'Referer': base_url})

    r = requests.get(url, headers=http_headers)
    print r.content  # => "<!DOCTYPE html><html><head>..."

Make sure to include the headers for each call after scraping for the cookies. Otherwise use a session and let cfscrape handle the cookies internally.