OpenSSL 1.1.1 message

Trying to scrape http://spacemov.top Until yesterday all was fine, but today i’m getting messages about captcha. Will upgrade to 1.1.1 solve this issue? Now i have 1.0.2q.

Output:

CloudflareCaptchaError: Cloudflare captcha challenge presented for spacemov.top (cfscrape cannot solve captchas). Your OpenSSL version is lower than 1.1.1. Please upgrade your OpenSSL library and recompile Python.

---

from fnccfscrapenodejs import CloudflareCaptchaError, create_scraper
scraper = create_scraper()
try:
    scraper.get('https://spacemov.top/')
except CloudflareCaptchaError as e:
    print(e.response.request.url)
    print(e.response.content)

Output here: https://paste.ofcode.org/wgUgx2AyPd5xwS7L3UDxuC

[ghost]

If not, you may elect to use TLSv1.1 at the cost of security. I do not recommend it but this is how you would do it.

import cfscrape
import ssl

from requests.adapters import HTTPAdapter
from urllib3.util.ssl_ import create_urllib3_context

# Using TLSv1.1 - non-modified ciphers list
class CustomAdapter(HTTPAdapter):
    def init_poolmanager(self, *args, **kwargs):
        ctx = create_urllib3_context(ssl.PROTOCOL_TLSv1)
        ctx.options &= ~ssl.OP_NO_TLSv1
        ctx.options &= ~ssl.OP_NO_TLSv1_1
        super(CustomAdapter, self).init_poolmanager(*args, ssl_context=ctx, **kwargs)

scraper = cfscrape.create_scraper()
scraper.mount('https://', CustomAdapter())

# Note that I use HTTPS here to avoid a redirect (HTTPS is forced regardless)
scraper.get('https://spacemov.top/')

[ghost]

Thanks for the reports. I think the problem is caused by SHA1 in the signature algorithms extension.

Signature algorithms hash: b10ec48d6fb7b3b35c76dfd53b3529bb

Signature algorithms as reported by ssllabs v1

(u'SHA512/RSA',
 u'SHA512/DSA',
 u'SHA512/ECDSA',
 u'SHA384/RSA',
 u'SHA384/DSA',
 u'SHA384/ECDSA',
 u'SHA256/RSA',
 u'SHA256/DSA',
 u'SHA256/ECDSA',
 u'SHA224/RSA',
 u'SHA224/DSA',
 u'SHA224/ECDSA',
 u'SHA1/RSA',
 u'SHA1/DSA',
 u'SHA1/ECDSA')

 u'SHA1/RSA',
 u'SHA1/DSA',
 u'SHA1/ECDSA'

I’m not sure that we can completely disable the use of SHA1 in the signature algorithms extension only. The only way to accomplish this from python is by indirectly passing a cipher list control string to SSL_CTX_set_cipher_list. When I attempt to do so it results in breakage…

Hopefully, updating openssl will resolve this problem for you.