SSL certificates not used by pipenv lock

I am trying to use pipenv at work and am running into SSL certificate errors despite my company’s certificates working in other applications, including pip and in requests:

Python 3.6.5 (default, Apr 14 2018, 13:17:30) 
[GCC 7.3.1 20180406] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get('https://pypi.org')
<Response [200]>
>>> 

pipenv is able to download packages just fine, but locking runs into SSL problems. I was directed to open a new issue after checking my TLS version (it is 1.2) here: https://github.com/pypa/pipenv/issues/591.

Please run $ python -m pipenv.help, and paste the results here.

$ python -m pipenv.help output

Pipenv version: '11.10.1'

Pipenv location: '/usr/lib/python3.6/site-packages/pipenv'

Python location: '/usr/bin/python'

Other Python installations in PATH:

• 3.6: /usr/bin/python3.6m

• 3.6: /usr/bin/python3.6

• 3.6: /bin/python3.6

• 3.6.5: /usr/bin/python

• 3.6.5: /bin/python

• 3.6.5: /usr/bin/python3

• 3.6.5: /bin/python3

PEP 508 Information:

{‘implementation_name’: ‘cpython’, ‘implementation_version’: ‘3.6.5’, ‘os_name’: ‘posix’, ‘platform_machine’: ‘x86_64’, ‘platform_python_implementation’: ‘CPython’, ‘platform_release’: ‘4.16.3-1-ARCH’, ‘platform_system’: ‘Linux’, ‘platform_version’: ‘#1 SMP PREEMPT Thu Apr 19 09:17:56 UTC 2018’, ‘python_full_version’: ‘3.6.5’, ‘python_version’: ‘3.6’, ‘sys_platform’: ‘linux’}

System environment variables:

• BROWSER
• COLORTERM
• CONDA_SHLVL
• DBUS_SESSION_BUS_ADDRESS
• DESKTOP_SESSION
• DISPLAY
• GDMSESSION
• GTK_MODULES
• HOME
• LANG
• LOGNAME
• LS_COLORS
• MAIL
• MOZ_PLUGIN_PATH
• PATH
• PWD
• SHELL
• SHLVL
• TERM
• USER
• VTE_VERSION
• WINDOWID
• XAUTHORITY
• XDG_CURRENT_DESKTOP
• XDG_GREETER_DATA_DIR
• XDG_RUNTIME_DIR
• XDG_SEAT
• XDG_SEAT_PATH
• XDG_SESSION_DESKTOP
• XDG_SESSION_ID
• XDG_SESSION_PATH
• XDG_SESSION_TYPE
• XDG_VTNR
• http_proxy
• https_proxy
• no_proxy
• PYTHONDONTWRITEBYTECODE
• PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /home/e330665/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
• SHELL: /usr/bin/fish
• LANG: en_US.UTF-8
• PWD: /home/e330665/sandbox/pipenv_test

---

Contents of Pipfile (‘/home/e330665/sandbox/pipenv_test/Pipfile’):

toml [[source]] url = “https://pypi.org/simple” verify_ssl = true name = “pypi”

[packages] requests = “*”

[dev-packages]

[requires] python_version = “3.6”

---

Expected result

Locking [dev-packages] dependencies…
Locking [packages] dependencies…
unicorns I guess

Actual result

Locking [dev-packages] dependencies…
Using pip: -i https://pypi.org/simple

                          ROUND 1                           
Current constraints:

Finding the best candidates:

Finding secondary dependencies:
------------------------------------------------------------
Result of round 1: stable, done

Locking [packages] dependencies…
Using pip: -i https://pypi.org/simple

                          ROUND 1                           
Current constraints:
  requests

Finding the best candidates:
  found candidate requests==2.18.4 (constraint was <any>)

Finding secondary dependencies:
  requests==2.18.4          requires certifi>=2017.4.17, chardet<3.1.0,>=3.0.2, idna<2.7,>=2.5, urllib3<1.23,>=1.21.1

New dependencies found in this round:
  adding ['certifi', '>=2017.4.17', '[]']
  adding ['chardet', '<3.1.0,>=3.0.2', '[]']
  adding ['idna', '<2.7,>=2.5', '[]']
  adding ['urllib3', '<1.23,>=1.21.1', '[]']
Removed dependencies in this round:
Unsafe dependencies in this round:
------------------------------------------------------------
Result of round 1: not stable

                          ROUND 2                           
Current constraints:
  certifi>=2017.4.17
  chardet<3.1.0,>=3.0.2
  idna<2.7,>=2.5
  requests
  urllib3<1.23,>=1.21.1

Finding the best candidates:
  found candidate certifi==2018.4.16 (constraint was >=2017.4.17)
  found candidate chardet==3.0.4 (constraint was >=3.0.2,<3.1.0)
  found candidate idna==2.6 (constraint was >=2.5,<2.7)
  found candidate requests==2.18.4 (constraint was <any>)
  found candidate urllib3==1.22 (constraint was >=1.21.1,<1.23)

Finding secondary dependencies:
  certifi==2018.4.16        requires -
  requests==2.18.4          requires certifi>=2017.4.17, chardet<3.1.0,>=3.0.2, idna<2.7,>=2.5, urllib3<1.23,>=1.21.1
  idna==2.6                 requires -
  chardet==3.0.4            requires -
  urllib3==1.22             requires -
------------------------------------------------------------
Result of round 2: stable, done

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/pipenv/vendor/urllib3/connectionpool.py", line 595, in urlopen
    self._prepare_proxy(conn)
  File "/usr/lib/python3.6/site-packages/pipenv/vendor/urllib3/connectionpool.py", line 816, in _prepare_proxy
    conn.connect()
  File "/usr/lib/python3.6/site-packages/pipenv/vendor/urllib3/connection.py", line 326, in connect
    ssl_context=context)
  File "/usr/lib/python3.6/site-packages/pipenv/vendor/urllib3/util/ssl_.py", line 329, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib64/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/usr/lib64/python3.6/ssl.py", line 814, in __init__
    self.do_handshake()
  File "/usr/lib64/python3.6/ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib64/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/pipenv/vendor/requests/adapters.py", line 440, in send
    timeout=timeout
  File "/usr/lib/python3.6/site-packages/pipenv/vendor/urllib3/connectionpool.py", line 639, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/lib/python3.6/site-packages/pipenv/vendor/urllib3/util/retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /pypi/chardet/json (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)'),))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/pipenv/resolver.py", line 85, in <module>
    main()
  File "/usr/lib/python3.6/site-packages/pipenv/resolver.py", line 74, in main
    system=system,
  File "/usr/lib/python3.6/site-packages/pipenv/resolver.py", line 65, in resolve
    allow_global=system,
  File "/usr/lib/python3.6/site-packages/pipenv/utils.py", line 514, in resolve_deps
    timeout=10,
  File "/usr/lib/python3.6/site-packages/pipenv/vendor/requests/sessions.py", line 521, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3.6/site-packages/pipenv/vendor/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3.6/site-packages/pipenv/vendor/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.6/site-packages/pipenv/vendor/requests/adapters.py", line 506, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /pypi/chardet/json (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)'),))

Steps to replicate

pipenv shell
pipenv install requests

[nateprewitt]

Setting the environment variable worked. I wonder why requests worked on its own without that set?

Hey @edwardreed81, sorry I missed this issue when it started. The problem you’re hitting is two separate versions of Requests. The one bundled with pipenv is Requests 2.11.1 which still had certificates packaged with Requests vs the Requests you have installed is (likely) Requests 2.18.4 which will detect certs on your machine. This is why you’re seeing variance between the two imports.

This was a fundamental change we made in Requests and is expected behaviour. We need to get our vendored copy of Requests updated in pipenv or simply removed if possible. We’ll look into getting the issue resolved tomorrow hopefully.

[techalchemy]

We bundle requests with our own PEM and so does pip. Try installing requests[security] as this may add relevant libraries