Cannot pip install a URL that redirects to a valid wheel

Environment

• pip version: 19.1.1
• Python version: 3.7
• OS: Linux

Description I attempted to pip install a URL which 302 redirects to a valid whl.

curl -v http://ezyang.com/nightly.php
*   Trying 2401:db00:20ff:ff70:face:b00c:0:1e10...
* TCP_NODELAY set
* Connected to fwdproxy (2401:db00:20ff:ff70:face:b00c:0:1e10) port 8080 (#0)
> GET http://ezyang.com/nightly.php HTTP/1.1
> Host: ezyang.com
> User-Agent: curl/7.59.0
> Accept: */*
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 302 Moved Temporarily
< Date: Wed, 31 Jul 2019 21:25:09 GMT
< Server: Apache
< X-Powered-By: PHP/5.5.26
< Location: https://download.pytorch.org/whl/nightly/torchaudio_nightly-0.4.0.dev20190731-cp37-cp37m-linux_x86_64.whl
< Content-Type: text/html
< Via: HTTP/1.1 18.4.86.46:80 (fwdproxy2/350 69.171.251.28)
< X-Connected-To: 18.4.86.46
< X-FB-IP-Type: allowed
< Connection: keep-alive
< Content-Length: 0
< 
* Connection #0 to host fwdproxy left intact

pip installing the redirected URL gets to dependency solving, as desired:

$ pip install https://download.pytorch.org/whl/nightly/torchaudio_nightly-0.4.0.dev20190731-cp37-cp37m-linux_x86_64.whl
Collecting torchaudio-nightly==0.4.0.dev20190731 from https://download.pytorch.org/whl/nightly/torchaudio_nightly-0.4.0.dev20190731-cp37-cp37m-linux_x86_64.whl
  Using cached https://download.pytorch.org/whl/nightly/torchaudio_nightly-0.4.0.dev20190731-cp37-cp37m-linux_x86_64.whl
Collecting torch-nightly==1.2.0.dev20190731 (from torchaudio-nightly==0.4.0.dev20190731)
  ERROR: Could not find a version that satisfies the requirement torch-nightly==1.2.0.dev20190731 (from torchaudio-nightly==0.4.0.dev20190731) (from versions: none)
ERROR: No matching distribution found for torch-nightly==1.2.0.dev20190731 (from torchaudio-nightly==0.4.0.dev20190731)

However, pip installing the redirecting URL fails earlier in a mysterious way.

$ pip install http://ezyang.com/nightly.php
Collecting http://ezyang.com/nightly.php
  Using cached http://ezyang.com/nightly.php
    ERROR: Complete output from command python setup.py egg_info:
    ERROR: Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/home/ezyang/miniconda3/envs/scratch1/lib/python3.7/tokenize.py", line 447, in open
        buffer = _builtin_open(filename, 'rb')
    FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-req-build-d0odg3he/setup.py'
    ----------------------------------------
ERROR: Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-d0odg3he/

Expected behavior I expect the second command to work.

[uranusjr]

pip relies on the URL to determine whether a package is wheel or not. Since the URL you passed does not end with .whl, it treats it as a source package. The redirect does not help since the check happens before the package is downloaded.

It is definitely technically possible to change this behaviour, but it would be quite involved, and I am not confident anyone would be willing to take on the task (it contributes to why I gave up on #6329 eventually, fwiw). At this point I’d choose to be practical and treat this as a feature™ (maybe document it explicitly) instead.

[ezyang]

In this case, an error saying, “Hey, the URL you passed me doesn’t look like a valid whl URL” would have been good.