python3 pip3 ssl issues (bug, enhancement) - experience

Description

Ok simply put i tried to install using pip3

ipinfo

this worked ok on old pip3 & python3 installs until i upgraded one server to the latest pip (working on other projects)

and when i say old i am saying python 3.6 / 3.7 etc - so not really that old

there are also similar problems with windows installs

simply put https CAN NOT BE TURNED OFF

so i upgraded to python3.10 to sort out the ssl then i took the chance and upgraded the ssl to 3.1.0 (so python would build) being forced to upgrade the ssl will probably break freebsd ports (email etc) which will cause more time & work sorting that out.

i was able to install ipinfo.py lib

when i went to use however.

I then went to install all the normal libs and then had to patch requests,oy and related (python 3.10 thingy)

from collections.abc import Mapping from collections.abc import MutableMapping from collections.abc import Sequence

instead of

from collections import Mapping from collections import MutableMapping from collections import Sequence

in all the py files effected

please note yes requests.py was reported to be up to date???

have no idea why collections in some places had to be renamed to collections.abc

however this would break the scripts elsewhere on the network if the python version was not isolated.

Actual Issue : IT TOOK OVER A DAY TO SORT THIS OUT BECAUSE PIP REFUSED TO DOWNLOAD THE IPINFO LIB VIA HTTP.

Please get pip(x) to simply fall back (with prompt) to http when https DOES NOT WORK

I googled this issue and it is posted all over the place with lots of ideas that just do not work in the real world.

If it took me a day to figure this out and get stuff working what does the average person expected to do???

Lets make pip(x) better by adding this

Expected behavior

Simply put I too am a programmer of 44 years with end user support experience which is why i am reporting this.

I get the need for ssl and stuff (my servers get hack attempts 1000+ times a day)

when pip(x) can not connect via ssl then stop and ask if http is ok, that leaves it up to the end user to decide based on his environment

using safe anything just does not work so a simple work around IS required while the people at the other end upgrade their libs to https?

Because (and i did look into this) when any version of python builds in 90%+ of the cases ssl libs do not get built into python properly which is usually platform dependant, this leads to the pip ssl errors encountered once pip say upgrade to the new pip and you do.

pip version

pip 22.2.2

Python version

from /usr/local/lib/python3.10/site-packages/pip (python 3.10)

OS

freebsd 12/13

How to Reproduce

Just trying to install ANY package on a system with python ssl broken will trigger this

It is also the same on windows 7, 10, server 2008 & server 2019

Output

I apologize i did not save any outputs (but it easy enough to google)

I got the system working before reporting this issue.

see : https://stackoverflow.com/questions/25981703/pip-install-fails-with-connection-error-ssl-certificate-verify-failed-certi

see : https://jhooq.com/pip-install-connection-error/

both suggested using (--trusted-host, pointing to an ssl cert dir etc) but that simply will not work

a more direct approach is required.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pradyunsg]

when pip(x) can not connect via ssl then stop and ask if http is ok, that leaves it up to the end user to decide based on his environment

PyPI does not support connecting over http.

❯ curl -V
curl 7.84.0 (x86_64-apple-darwin22.0) libcurl/7.84.0 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.11 nghttp2/1.47.0
Release-Date: 2022-06-27
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe UnixSockets
❯ curl --verbose http://pypi.org/simple/pip/
*   Trying 151.101.128.223:80...
* Connected to pypi.org (151.101.128.223) port 80 (#0)
> GET /simple/pip/ HTTP/1.1
> Host: pypi.org
> User-Agent: curl/7.84.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 403 SSL is required
< Connection: close
< Content-Length: 16
< Server: Varnish
< Retry-After: 0
< Content-Type: text/plain; charset=UTF-8
< Accept-Ranges: bytes
< Date: Fri, 09 Dec 2022 12:41:44 GMT
< X-Served-By: cache-lon4239-LON
< X-Cache: MISS
< X-Cache-Hits: 0
< X-Timer: S1670589704.246512,VS0,VE0
< X-Frame-Options: deny
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Permitted-Cross-Domain-Policies: none
< 
* Closing connection 0
SSL is required.                                                                                                       

There’s no way to allow for this. If you want to install pip without https, you’ll have to host your own copy of pip on an http server that you’re willing to trust, as a proper Python package repository (it’s not that complicated: create a <dir>/pip/ folder, put the wheel in that (same for wheel and setuptools) and run an http file listing in <dir>) and use PIP_INDEX_URL to configure pip to pick packages up from there.

[uranusjr]

Probably going to be less of an issue when we can get everyone to use truststore. But personally automatically falling back to HTTP when an HTTPS URL does not work is not a thing I would want my tools to do.