Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

OpenSSL unsafe legacy renegotiation disabled error

See original GitHub issue

Description

I get an SSL issue on a working site twisted.web._newclient.ResponseNeverReceived: [<twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines', '', 'unsafe legacy renegotiation disabled')]>]

Steps to Reproduce

scrapy shell https://dorotheum.com

Expected behavior: HTML page

Actual behavior: the error above

Reproduces how often: 100%

Versions

Scrapy       : 2.6.1
lxml         : 4.8.0.0
libxml2      : 2.9.4
cssselect    : 1.1.0
parsel       : 1.6.0
w3lib        : 1.22.0
Twisted      : 22.4.0
Python       : 3.9.12 (main, Mar 26 2022, 15:44:31) - [Clang 13.1.6 (clang-1316.0.21.2)]
pyOpenSSL    : 22.0.0 (OpenSSL 3.0.3 3 May 2022)
cryptography : 37.0.2
Platform     : macOS-12.2.1-arm64-arm-64bit

Issue Analytics

State:
Created a year ago
Reactions:1
Comments:17 (7 by maintainers)

Top GitHub Comments

6reactions

HASKADOGcommented, May 6, 2022

@wRAR fixed by downgrading cryptography to 36.0.2 Here’s my current scrapy version --verbose result

Scrapy       : 2.6.1
lxml         : 4.8.0.0
libxml2      : 2.9.4
cssselect    : 1.1.0
parsel       : 1.6.0
w3lib        : 1.22.0
Twisted      : 22.4.0
Python       : 3.9.12 (main, Mar 26 2022, 15:44:31) - [Clang 13.1.6 (clang-1316.0.21.2)]
pyOpenSSL    : 22.0.0 (OpenSSL 1.1.1n  15 Mar 2022)
cryptography : 36.0.2
Platform     : macOS-12.3.1-arm64-arm-64bit

Now pyOpenSSL uses OpenSSL 1.1.1n Even though the problem is fixed, the flags enhancement would be great. Thank you so much @wRAR !

4reactions

aysegulccommented, Oct 10, 2022

I ran into the same problem, and built a custom context factory to solve it. I hope it will be helpful for others:

contextfactory.py (within MyProject folder)

from scrapy.core.downloader.contextfactory import ScrapyClientContextFactory


class LegacyConnectContextFactory(ScrapyClientContextFactory):

    def getContext(self, hostname=None, port=None):
        ctx = self.getCertificateOptions().getContext()
        ctx.set_options(0x4)
        return ctx

within my_spider.py

class MySpider(Spider):
    name = "my_spider"

    custom_settings = {
        'DOWNLOADER_CLIENTCONTEXTFACTORY': 'MyProject.contextfactory.LegacyConnectContextFactory',
    }

Top Results From Across the Web

SSL error unsafe legacy renegotiation disabled - Stack Overflow

It described how to easily disable SSL verification through /etc/openssl.conf for all applications (and all systems). When done in a regulated ...

OpenSSL issue: "unsafe legacy renegotiation disabled" #4543

Insomnia was nice while it lasted, but it's unusable if there isn't a way to ignore the SSL errors.

Openssl Enable Legacy Renegotiation - awk

If you get this error, your openssl binaries are compiled with legacy renegotiation disabled by default. This disables any non TLS 1.3 libraries ......

SSL routines::unsafe legacy renegotiation disabled

We're getting issues in the build pipeline where OpenSSL3 is failing to connect through the proxy. We get the error unsafe legacy renegotiation...

Curl Panics unsafe legacy renegotiation disabled - General

... [error:0A000152:SSL routines::unsafe legacy renegotiation disabled] Error: Failed to download metadata for repo 'baseos': Cannot prepare ...