sslv3 alert handshake failure when making a request

Hi there, I recently upgraded to the latest scrapy and on some sites SSL enabled sites I get an exception when trying to make requests to it, while on previous scrapy versions I didn’t have this issue. The issue can be seen by making a request with scrapy shell:

scrapy shell "https://www.gohastings.com/"

The error I get is: Retrying <GET https://www.gohastings.com/> (failed 1 times): <twisted.python.failure.Failure OpenSSL.SSL.Error: ('SSL routines', 'SSL3_READ_BYTES', 'sslv3 alert handshake failure'), ('SSL routines', 'SSL3_WRITE_BYTES', 'ssl handshake failure')>

[kneufeld]

Well I have no idea what’s going on now.

I’ve uninstalled cryptography and pyopenssl and reinstalled them with and without brew prefix flags and pOpenSSL is linking to the newer homebrew ssl versions and everything is working.

So, to anybody who’s hitting this bug:

scrapy version -v | grep pyOpenSSL

will return one of the following:

pyOpenSSL : 16.0.0 (OpenSSL 1.0.2g  1 Mar 2016)
pyOpenSSL : 16.0.0 (OpenSSL 0.9.8zg 14 July 2015)

If it’s OpenSSL version 0.9.8 then you need to try again.

rm -rf ~/Library/Caches/pip
pip uninstall cryptography pyopenssl
env LDFLAGS="-L$(brew --prefix openssl)/lib" CFLAGS="-I$(brew --prefix openssl)/include" pip install cryptography pyopenssl

So basically you can probably ignore my previous two posts.

[kneufeld]

Here ya go: https://github.com/scrapy/scrapy/pull/1909