SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] with python requests library

Steps to reproduce the problem:

import requests
import urllib.request
with urllib.request.urlopen('https://facebook.com/') as response:
    print(len(response.read()))
print(len(requests.get('https://facebook.com').text))

Without mitm proxy:

206541
206809

With mitm proxy:

206922
---------------------------------------------------------------------------
SSLError                                  Traceback (most recent call last)
~/Desktop/proj/.venv/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, **response_kw)
    553             if is_new_proxy_conn:
--> 554                 self._prepare_proxy(conn)
    555 

~/Desktop/proj/.venv/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py in _prepare_proxy(self, conn)
    749 
--> 750         conn.connect()
    751 

~/Desktop/proj/.venv/lib/python3.6/site-packages/requests/packages/urllib3/connection.py in connect(self)
    251                                     server_hostname=hostname,
--> 252                                     ssl_version=resolved_ssl_version)
    253 

~/Desktop/proj/.venv/lib/python3.6/site-packages/requests/packages/urllib3/util/ssl_.py in ssl_wrap_socket(sock, keyfile, certfile, cert_reqs, ca_certs, server_hostname, ssl_version, ciphers, ssl_context, ca_cert_dir)
    304     if HAS_SNI:  # Platform-specific: OpenSSL with enabled SNI
--> 305         return context.wrap_socket(sock, server_hostname=server_hostname)
    306 

/usr/lib/python3.6/ssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session)
    400                          server_hostname=server_hostname,
--> 401                          _context=self, _session=session)
    402 

/usr/lib/python3.6/ssl.py in __init__(self, sock, keyfile, certfile, server_side, cert_reqs, ssl_version, ca_certs, do_handshake_on_connect, family, type, proto, fileno, suppress_ragged_eofs, npn_protocols, ciphers, server_hostname, _context, _session)
    807                         raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
--> 808                     self.do_handshake()
    809 

/usr/lib/python3.6/ssl.py in do_handshake(self, block)
   1060                 self.settimeout(None)
-> 1061             self._sslobj.do_handshake()
   1062         finally:

/usr/lib/python3.6/ssl.py in do_handshake(self)
    682         """Start the SSL/TLS handshake."""
--> 683         self._sslobj.do_handshake()
    684         if self.context.check_hostname:

SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:748)

During handling of the above exception, another exception occurred:

SSLError                                  Traceback (most recent call last)
~/Desktop/proj/.venv/lib/python3.6/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    375                     retries=self.max_retries,
--> 376                     timeout=timeout
    377                 )

~/Desktop/proj/.venv/lib/python3.6/site-packages/requests/packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, **response_kw)
    587             release_conn = True
--> 588             raise SSLError(e)
    589 

SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:748)

During handling of the above exception, another exception occurred:

SSLError                                  Traceback (most recent call last)
<ipython-input-58-20b3921db13c> in <module>()
      3 with urllib.request.urlopen('https://facebook.com/') as response:
      4     print(len(response.read()))
----> 5 print(len(requests.get('https://facebook.com').text))

~/Desktop/proj/.venv/lib/python3.6/site-packages/requests/api.py in get(url, params, **kwargs)
     65 
     66     kwargs.setdefault('allow_redirects', True)
---> 67     return request('get', url, params=params, **kwargs)
     68 
     69 

~/Desktop/proj/.venv/lib/python3.6/site-packages/requests/api.py in request(method, url, **kwargs)
     51     # cases, and look like a memory leak in others.
     52     with sessions.Session() as session:
---> 53         return session.request(method=method, url=url, **kwargs)
     54 
     55 

~/Desktop/proj/.venv/lib/python3.6/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    466         }
    467         send_kwargs.update(settings)
--> 468         resp = self.send(prep, **send_kwargs)
    469 
    470         return resp

~/Desktop/proj/.venv/lib/python3.6/site-packages/requests/sessions.py in send(self, request, **kwargs)
    574 
    575         # Send the request
--> 576         r = adapter.send(request, **kwargs)
    577 
    578         # Total elapsed time of the request (approximately)

~/Desktop/proj/.venv/lib/python3.6/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    445         except (_SSLError, _HTTPError) as e:
    446             if isinstance(e, _SSLError):
--> 447                 raise SSLError(e, request=request)
    448             elif isinstance(e, ReadTimeoutError):
    449                 raise ReadTimeout(e, request=request)

SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:748)

Any other comments? What have you tried so far?

It’s visible from the logs using urllib works alright, but requests fail miserably.

this is output of pip freeze:

aiodns==1.1.1
aiohttp==2.2.3
async-timeout==1.2.1
asyncio==3.4.3
beautifulsoup4==4.6.0
bleach==2.0.0
cchardet==2.1.1
ccxt==1.4.12
certifi==2017.7.27.1
chardet==3.0.4
click==6.7
colorama==0.3.9
coloredlogs==7.1
ConfigArgParse==0.12.0
coverage==4.4.1
Cython==0.26
dash==0.17.7
dash-core-components==0.5.3
dash-html-components==0.6.2
dash-renderer==0.7.3
decorator==4.1.2
entrypoints==0.2.3
Flask==0.12.2
Flask-Compress==1.4.0
Flask-SeaSurf==0.2.2
forex-python==0.3.2
gevent==1.2.2
graypy==0.2.14
greenlet==0.4.12
gunicorn==19.7.1
html5lib==0.999999999
humanfriendly==4.4
idna==2.6
ipykernel==4.6.1
ipython==6.1.0
ipython-genutils==0.2.0
ipywidgets==6.0.0
itsdangerous==0.24
jedi==0.10.2
Jinja2==2.9.6
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.1.0
jupyter-console==5.1.0
jupyter-core==4.3.0
logging-tree==1.7
lxml==3.8.0
MarkupSafe==1.0
mistune==0.7.4
multidict==3.1.1
nbconvert==5.2.1
nbformat==4.3.0
nose==1.3.7
nose2==0.6.5
notebook==5.0.0
numpy==1.13.1
overrides==1.8
pandas==0.20.3
pandocfilters==1.4.1
pexpect==4.2.1
pg8000==1.10.6
pickleshare==0.7.4
plotly==2.0.14
prompt-toolkit==1.0.14
prompter==0.3.10
psycopg2==2.7.1
ptyprocess==0.5.2
py==1.4.34
pycares==2.2.0
Pygments==2.2.0
pytest==3.2.1
python-dateutil==2.6.1
pytz==2017.2
PyYAML==3.12
pyzmq==16.0.2
qtconsole==4.3.0
requests==2.9.0
sh==1.12.14
simplegeneric==0.8.1
simplejson==3.11.1
six==1.10.0
slack-logger==0.2.0
slackclient==1.0.6
SQLAlchemy==1.1.13
SQLAlchemy-Utils==0.32.14
terminado==0.6
testpath==0.3.1
toolz==0.8.2
tornado==4.5.1
traitlets==4.3.2
urllib3==1.22
uvloop==0.8.0
voluptuous==0.10.5
wcwidth==0.1.7
webencodings==0.5.1
websocket==0.2.1
websocket-client==0.44.0
websockets==3.3
Werkzeug==0.12.2
widgetsnbextension==2.0.0
yarl==0.11.0

System information

Mitmproxy version: 2.0.2 (release version) Python version: 3.6.2 Platform: Linux-4.12.6-1-ARCH-x86_64-with-arch SSL version: OpenSSL 1.1.0f 25 May 2017 Linux distro: arch

[madux]

THE SIMPLE SOLUTION FROM Maduka Sopulu IS TO ADD verify=False

eg. response = requests.post(url, data = parameters, verify=False)

For any requests type, add verify=False

[felixhummel]

Thanks! I had the same problem.

Just for future reference: If you are on Debian / Ubuntu, and want requests to use your system bundle, simply run:

REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt python

This list of trusted CAs can also be specified through the REQUESTS_CA_BUNDLE environment variable.