question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

unable to get local issuer certificate for files.pythonhosted.org

See original GitHub issue

Environment

  • pip version: 20.0.2
  • Python version: 3.7.6, provided via macbrew (i.e. brew install python)
  • OS: OS X 10.15.2

Description

I’m suddenly and inexplicably unable to install/upgrade anything from PyPI. I’ve had a solid dev environment for months and I can’t think of what’s changed (in the shell) — The only thing that has changed is that I’ve been traveling and staying in hotels with WIFI connection agreement pages. (Could that cause all of this???) I’ve also tried connecting by tethering to my cellphone, but without success.

pip3 install <foo> results in '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1076)'

Full example here
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None,
status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1,
'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local
issuer certificate (_ssl.c:1076)'))':
/s/files.pythonhosted.org/packages/13/b4/3ffa55f77161cff9a5220f162670f7c5eb00df52e00939e203f601b0f579/pipenv-2018.11.26-py3-none-any.whl?X-OpenDNS-Session=_4e248e6b04d820461e08dfb05ee293a9382f9270fe56_4uXwMBrs_
... 
ERROR: Could not install packages due to an EnvironmentError:
HTTPSConnectionPool(host='files.pythonhosted.org.x.4e248e6b04d820461e08dfb05ee293a9382f.9270fe56.id.opendns.com',
port=443): Max retries exceeded with url:
/s/files.pythonhosted.org/packages/13/b4/3ffa55f77161cff9a5220f162670f7c5eb00df52e00939e203f601b0f579/pipenv-2018.11.26-py3-none-any.whl?X-OpenDNS-Session=_4e248e6b04d820461e08dfb05ee293a9382f9270fe56_4uXwMBrs_
(Caused by SSLError(SSLCertVerificationError(1, '[SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer
certificate (_ssl.c:1076)')))

Adding --trusted-host=files.pythonhosted.org and/or --trusted-host=files.pythonhosted.org:443 has no effect.

Here’s the debugging info that was suggested in similar issue #6915 – seems all good.

$ openssl s_client -connect pypi.org:443 -showcerts
CONNECTED(00000005)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert High Assurance EV Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 Extended Validation Server CA
verify return:1
depth=0 businessCategory = Private Organization, jurisdictionCountryName = US, jurisdictionStateOrProvinceName = Delaware, serialNumber = 3359300, C = US, ST = New Hampshire, L = Wolfeboro, O = Python Software Foundation, CN = www.python.org
verify return:1
---
Certificate chain
 0 s:/businessCategory=Private Organization/jurisdictionCountryName=US/jurisdictionStateOrProvinceName=Delaware/serialNumber=3359300/C=US/ST=New Hampshire/L=Wolfeboro/O=Python Software Foundation/CN=www.python.org
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended Validation Server CA
full certs here
$ openssl s_client -connect pypi.org:443 -showcerts
CONNECTED(00000005)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert High Assurance EV Root CA
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 Extended Validation Server CA
verify return:1
depth=0 businessCategory = Private Organization, jurisdictionCountryName = US, jurisdictionStateOrProvinceName = Delaware, serialNumber = 3359300, C = US, ST = New Hampshire, L = Wolfeboro, O = Python Software Foundation, CN = www.python.org
verify return:1
---
Certificate chain
 0 s:/businessCategory=Private Organization/jurisdictionCountryName=US/jurisdictionStateOrProvinceName=Delaware/serialNumber=3359300/C=US/ST=New Hampshire/L=Wolfeboro/O=Python Software Foundation/CN=www.python.org
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended Validation Server CA
-----BEGIN CERTIFICATE-----
MIIIczCCB1ugAwIBAgIQDl7PGBeDAG2brEU2EfVJEjANBgkqhkiG9w0BAQsFADB1
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMTQwMgYDVQQDEytEaWdpQ2VydCBTSEEyIEV4dGVuZGVk
IFZhbGlkYXRpb24gU2VydmVyIENBMB4XDTE4MDkxODAwMDAwMFoXDTIwMTAxNDEy
MDAwMFowgdgxHTAbBgNVBA8MFFByaXZhdGUgT3JnYW5pemF0aW9uMRMwEQYLKwYB
BAGCNzwCAQMTAlVTMRkwFwYLKwYBBAGCNzwCAQITCERlbGF3YXJlMRAwDgYDVQQF
EwczMzU5MzAwMQswCQYDVQQGEwJVUzEWMBQGA1UECBMNTmV3IEhhbXBzaGlyZTES
MBAGA1UEBxMJV29sZmVib3JvMSMwIQYDVQQKExpQeXRob24gU29mdHdhcmUgRm91
bmRhdGlvbjEXMBUGA1UEAxMOd3d3LnB5dGhvbi5vcmcwggEiMA0GCSqGSIb3DQEB
AQUAA4IBDwAwggEKAoIBAQD2roGTDc+m3g3mReIf7j5rzLovnk3Zggvy2kWWMnDB
Yz2m6sgxOL1pkzIu7YNe4noH7ZRym0OIQjZbDIleB7SOGAoD2BBv+Mv79HtedJ3d
vhXwmgRenlNxBxhDcaVLDONVC9Ir3Ft46uuGIrXnKB8L1KZV6h8IFC2G3GZXJtkw
EojXdmJFzuRrKMRi0cLPlF60upRISIj3jg9kK4D+f0xYrsQAJvK0veqRsNQ8bXPE
YR8kG4Paj7rOeh3FVGxrKOKNpoI+kV6EqOVIhJY698P7EbDLGjG2Im9P6VFmwXod
YAZ2CTFaMlrC0FljAO/FKeQKR29vitthLMutcd+AkoDDAgMBAAGjggSZMIIElTAf
BgNVHSMEGDAWgBQ901Cl1qCt7vNKYApl0yHU+PjWDzAdBgNVHQ4EFgQUUTsyHAXZ
y6d2HWn+8MZNUhBCkEwwggFCBgNVHREEggE5MIIBNYIOd3d3LnB5dGhvbi5vcmeC
D2RvY3MucHl0aG9uLm9yZ4IPYnVncy5weXRob24ub3Jngg93aWtpLnB5dGhvbi5v
cmeCDWhnLnB5dGhvbi5vcmeCD21haWwucHl0aG9uLm9yZ4IPcHlwaS5weXRob24u
b3JnghRwYWNrYWdpbmcucHl0aG9uLm9yZ4IQbG9naW4ucHl0aG9uLm9yZ4ISZGlz
Y3Vzcy5weXRob24ub3Jnggx1cy5weWNvbi5vcmeCB3B5cGkuaW+CDGRvY3MucHlw
aS5pb4IIcHlwaS5vcmeCDWRvY3MucHlwaS5vcmeCD2RvbmF0ZS5weXBpLm9yZ4IT
ZGV2Z3VpZGUucHl0aG9uLm9yZ4ITd3d3LmJ1Z3MucHl0aG9uLm9yZ4IKcHl0aG9u
Lm9yZzAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUF
BwMCMHUGA1UdHwRuMGwwNKAyoDCGLmh0dHA6Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9z
aGEyLWV2LXNlcnZlci1nMi5jcmwwNKAyoDCGLmh0dHA6Ly9jcmw0LmRpZ2ljZXJ0
LmNvbS9zaGEyLWV2LXNlcnZlci1nMi5jcmwwSwYDVR0gBEQwQjA3BglghkgBhv1s
AgEwKjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cuZGlnaWNlcnQuY29tL0NQUzAH
BgVngQwBATCBiAYIKwYBBQUHAQEEfDB6MCQGCCsGAQUFBzABhhhodHRwOi8vb2Nz
cC5kaWdpY2VydC5jb20wUgYIKwYBBQUHMAKGRmh0dHA6Ly9jYWNlcnRzLmRpZ2lj
ZXJ0LmNvbS9EaWdpQ2VydFNIQTJFeHRlbmRlZFZhbGlkYXRpb25TZXJ2ZXJDQS5j
cnQwDAYDVR0TAQH/BAIwADCCAX8GCisGAQQB1nkCBAIEggFvBIIBawFpAHcA7ku9
t3XOYLrhQmkfq+GeZqMPfl+wctiDAMR7iXqo/csAAAFl7l3MWwAABAMASDBGAiEA
qfAJSfOHG4r8YvzTkZsr8cEXFcnFIns40+JXVdgY0vsCIQCvnB+YExtMRQVQXONc
glOTsIYmNgYPrtyHYsTj/Xua5QB3AFYUBpov18Ls0/XhvUSyPsdGdrm8mRFcwO+U
mFXWidDdAAABZe5dzHoAAAQDAEgwRgIhAIF6xXakKVdREciK8aM2z1c71eiU8qF/
UCZbl4sEfLTQAiEA870pR9Hazod3FgZszr9itk8sYPLoQjV9/WENl0HXXGwAdQC7
2d+8H4pxtZOUI5eqkntHOFeVCqtS6BqQlmQ2jh7RhQAAAWXuXcwzAAAEAwBGMEQC
IAeIvcaqPATJrCo3+ceBlVbsyPJcDPM7QGLpaRPFBduQAiBHNfcsBLaelS9bUmfU
R0VjJLoTA39AJAj8oU6iAzruHDANBgkqhkiG9w0BAQsFAAOCAQEAwH5+PXougfo5
qMVIE65Dei/CEb4ahZfoMvvlJRvuNAI/ARWYQSHh6IKXdxFxIE5hCC/NNtdAcc1p
CoZ2+IM/x0cGBjHZegSjhXfQy+w7twfgyeTSNalV2jzKQ0Yv/JvfI9qiMdhEQbfL
qaQ6Nj/js8uvQqWf6w8yo7hAzKs1jTF7Wy/cM0lvqNocDWYROhAVcI+jSMKwlcLv
75xAbOZqw2D483mkQizVj7wQ1fYP3Tr0wfvFNeoIAPUXLrEHiEmHWA0h+U9KEgVf
VQ3PvzItecKWFI+0d9RpNupc1HdipBCySvnNa1LcG2AviYXpIsbo8EBvbj0HcEVx
7NEANHNX2g==
-----END CERTIFICATE-----
 1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended Validation Server CA
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
-----BEGIN CERTIFICATE-----
MIIEtjCCA56gAwIBAgIQDHmpRLCMEZUgkmFf4msdgzANBgkqhkiG9w0BAQsFADBs
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMSswKQYDVQQDEyJEaWdpQ2VydCBIaWdoIEFzc3VyYW5j
ZSBFViBSb290IENBMB4XDTEzMTAyMjEyMDAwMFoXDTI4MTAyMjEyMDAwMFowdTEL
MAkGA1UEBhMCVVMxFTATBgNVBAoTDERpZ2lDZXJ0IEluYzEZMBcGA1UECxMQd3d3
LmRpZ2ljZXJ0LmNvbTE0MDIGA1UEAxMrRGlnaUNlcnQgU0hBMiBFeHRlbmRlZCBW
YWxpZGF0aW9uIFNlcnZlciBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
ggEBANdTpARR+JmmFkhLZyeqk0nQOe0MsLAAh/FnKIaFjI5j2ryxQDji0/XspQUY
uD0+xZkXMuwYjPrxDKZkIYXLBxA0sFKIKx9om9KxjxKws9LniB8f7zh3VFNfgHk/
LhqqqB5LKw2rt2O5Nbd9FLxZS99RStKh4gzikIKHaq7q12TWmFXo/a8aUGxUvBHy
/Urynbt/DvTVvo4WiRJV2MBxNO723C3sxIclho3YIeSwTQyJ3DkmF93215SF2AQh
cJ1vb/9cuhnhRctWVyh+HA1BV6q3uCe7seT6Ku8hI3UarS2bhjWMnHe1c63YlC3k
8wyd7sFOYn4XwHGeLN7x+RAoGTMCAwEAAaOCAUkwggFFMBIGA1UdEwEB/wQIMAYB
Af8CAQAwDgYDVR0PAQH/BAQDAgGGMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEF
BQcDAjA0BggrBgEFBQcBAQQoMCYwJAYIKwYBBQUHMAGGGGh0dHA6Ly9vY3NwLmRp
Z2ljZXJ0LmNvbTBLBgNVHR8ERDBCMECgPqA8hjpodHRwOi8vY3JsNC5kaWdpY2Vy
dC5jb20vRGlnaUNlcnRIaWdoQXNzdXJhbmNlRVZSb290Q0EuY3JsMD0GA1UdIAQ2
MDQwMgYEVR0gADAqMCgGCCsGAQUFBwIBFhxodHRwczovL3d3dy5kaWdpY2VydC5j
b20vQ1BTMB0GA1UdDgQWBBQ901Cl1qCt7vNKYApl0yHU+PjWDzAfBgNVHSMEGDAW
gBSxPsNpA/i/RwHUmCYaCALvY2QrwzANBgkqhkiG9w0BAQsFAAOCAQEAnbbQkIbh
hgLtxaDwNBx0wY12zIYKqPBKikLWP8ipTa18CK3mtlC4ohpNiAexKSHc59rGPCHg
4xFJcKx6HQGkyhE6V6t9VypAdP3THYUYUN9XR3WhfVUgLkc3UHKMf4Ib0mKPLQNa
2sPIoc4sUqIAY+tzunHISScjl2SFnjgOrWNoPLpSgVh5oywM395t6zHyuqB8bPEs
1OG9d4Q3A84ytciagRpKkk47RpqF/oOi+Z6Mo8wNXrM9zwR4jxQUezKcxwCmXMS1
oVWNWlZopCJwqjyBcdmdqEU79OX2olHdx3ti6G8MdOu42vi/hw15UJGQmxg7kVkn
8TUoE6smftX3eg==
-----END CERTIFICATE-----
---
Server certificate
subject=/businessCategory=Private Organization/jurisdictionCountryName=US/jurisdictionStateOrProvinceName=Delaware/serialNumber=3359300/C=US/ST=New Hampshire/L=Wolfeboro/O=Python Software Foundation/CN=www.python.org
issuer=/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 Extended Validation Server CA
---
No client certificate CA names sent
Server Temp Key: ECDH, X25519, 253 bits
---
SSL handshake has read 4001 bytes and written 289 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: A60B894B0F94EB86EF02B1EBFB596108EEDEC2E4D6A1817E025D0D319BF98C81
    Session-ID-ctx: 
    Master-Key: FF23A786EAC7D28D4D1DADFB1E9D7466FC9B27590E4197B410F46E8F87D5E6B94AE285FC932A600A1293B34A1D98C724
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - b1 90 43 58 05 c9 7e e5-a1 bd ad c6 9a f7 e4 19   ..CX..~.........
    0010 - 13 0e 0d 3e ec c6 b2 ef-2a d2 44 f7 e0 e9 cf 80   ...>....*.D.....
    0020 - 92 b0 8f 0e d6 dd b2 84-36 d8 21 ee c4 13 e5 a0   ........6.!.....
    0030 - 90 c3 6e 0d cb 22 29 3a-15 66 cb ef 84 ee 1c f0   ..n.."):.f......
    0040 - 4e b7 87 ea 5a b3 cf 20-0f 73 d4 16 14 40 33 b8   N...Z.. .s...@3.
    0050 - 79 37 49 b8 51 fb 0d bb-a4 77 e0 07 ce 36 52 9f   y7I.Q....w...6R.
    0060 - 7b 90 8e cb 4c b3 99 23-53 18 2b 00 8d 7e 82 54   {...L..#S.+..~.T
    0070 - 1b ee 96 e1 af 6f db 7b-87 40 c6 b8 60 53 6c 0b   .....o.{.@..`Sl.
    0080 - 2d a5 5a 7c 92 e1 a4 d8-69 4c b1 b4 a6 b6 54 1d   -.Z|....iL....T.
    0090 - 82 a2 70 de 9f 4b 61 62-5a 47 ca 8d 20 e2 b5 ab   ..p..KabZG.. ...

    Start Time: 1580179437
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---

But when I try with files.pythonhosted.org I get an error:

$ openssl s_client -connect files.pythonhosted.org:443 -showcerts
CONNECTED(00000006)
depth=2 C = US, ST = California, L = San Francisco, O = Cisco, CN = Cisco Umbrella Primary SubCA
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
 0 s:/C=US/ST=California/L=San Francisco/O=Cisco Systems, Inc./CN=*.opendns.com
   i:/O=Cisco/CN=Cisco Umbrella Secondary SubCA lax-SG

And explicitly passing the certifi.pem file to openssl doesn’t help:

$ python3 -c 'import certifi; print(certifi.where())'
/usr/local/lib/python3.7/site-packages/certifi/cacert.pem
$ openssl s_client -CAfile /usr/local/lib/python3.7/site-packages/certifi/cacert.pem -connect files.pythonhosted.org:443
CONNECTED(00000006)
depth=2 C = US, ST = California, L = San Francisco, O = Cisco, CN = Cisco Umbrella Primary SubCA
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
 0 s:/C=US/ST=California/L=San Francisco/O=Cisco Systems, Inc./CN=*.opendns.com
   i:/O=Cisco/CN=Cisco Umbrella Secondary SubCA lax-SG
 1 s:/O=Cisco/CN=Cisco Umbrella Secondary SubCA lax-SG
   i:/C=US/ST=California/L=San Francisco/O=Cisco/CN=Cisco Umbrella Primary SubCA
 2 s:/C=US/ST=California/L=San Francisco/O=Cisco/CN=Cisco Umbrella Primary SubCA
   i:/O=Cisco/CN=Cisco Umbrella Root CA
---

Expected behavior These pip3 install commands have always worked for me in the past.

How to Reproduce I figure something is kooky with my environment, so it may be hard to reproduce this. I have completely uninstalled and reinstalled my python3 (provided by macbrew) and I still get the error. Any help or pointers much appreciated. Apologies if this is off-topic for this repo, but based on the helpful response to #6915, I thought I’d make an appeal. Thank you.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:4
  • Comments:44 (14 by maintainers)

github_iconTop GitHub Comments

5reactions
odoublewencommented, Jan 28, 2020

I figured something out. When I am connected to my company VPN, everything Just Works. (No matter what wifi I am using.) The most obvious difference is the nslookup – now there is a real IP for the DNS, rather than the loopback 127.0.0.1. (I am obfuscating the actual IP below):

$ nslookup files.pythonhosted.org
Server:		123.45.67.89  [obfuscated]
Address:	123.45.67.89#53

Non-authoritative answer:
files.pythonhosted.org	canonical name = dualstack.r.ssl.global.fastly.net.
Name:	dualstack.r.ssl.global.fastly.net
Address: 151.101.1.63
Name:	dualstack.r.ssl.global.fastly.net
Address: 151.101.65.63
Name:	dualstack.r.ssl.global.fastly.net
Address: 151.101.129.63
Name:	dualstack.r.ssl.global.fastly.net
Address: 151.101.193.63

Not sure why I don’t get proper NS lookup when not on company VPN, but now I have a way forward so I don’t need to bother you any more.

Thanks very much Chris and sorry to bother you with my hair pulling!

4reactions
random-langcommented, May 12, 2020

As a corporate security guy, this certainly is normal behaviour.

Cisco Umbrella (ne OpenDNS) uses selective proxying for sites that have unusual access patterns.

This would not be an issue if Pip by default checked the local certificate store of the corporate device rather than using a different list.

Suggest you either mark this as not a bug or adjust to always use the local cert store, which should contain the corps trusted CAs (and will certainly contain the Umbrella root CA if the corp uses Umbrealla)

Read more comments on GitHub >

github_iconTop Results From Across the Web

installing packages with pip returns error [duplicate]
Try this: C:\Users\riley> pip install --trusted-host pypi.org numpy. More info here.
Read more >
Cannot update or install anything with pip, SSL error
... failed: unable to get local issuer certificate (_ssl.c:1045)'))': ... host pypi.python.org - - trusted - host files.pythonhosted.org ...
Read more >
How to fix - Python pip install connection error SSL ...
In this article, we are going to see the error connection error SSL CERTIFICATE_VERIFY_FAILED certificate verify failed (_ssl.c:598) which ...
Read more >
PIP connection Error : SSL CERTIFICATE VERIFY FAILED
The most common issue in installing python package in a company's network is failure of verification of SSL Certificate. Sometimes company blocks some...
Read more >
HTTPS Certificates - pip documentation v22.3.1
By default, pip will perform SSL certificate verification for network connections ... “certificate verify failed: unable to get local issuer certificate”:.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found