pip install does not support 303 (See Other) as valid cacheable response

Environment

• pip version: 20.1.1
• Python version: Python 3.6.10 x64
• OS: Ubuntu 18.04.4 LTS

Description We have a private packages host that provides capabilities to install python packages. Along the way, the download URL that returns a 303 to the client with a signed URL that is then used as the url for download. Because pip does not save the original download URL, there is always a cache miss in pip resulting in longer build times.

Expected behavior pip’s caching happens at a per-http call level. In other words, pip’s cache is Dictionary<Uri,Response>. If pip’s cache were a Dictionary<PackageIdentity,byte[]>, then this problem would not occur.

How to Reproduce requirements.txt has a single numpy v.1.18.1 python -m pip install --upgrade pip wheel && pip install -v -r requirements.txt

Output

Created temporary directory: /tmp/pip-unpack-7o8lnaf3 

 Looking up "https://<host>/<downloadURL>/pypi/download/numpy/1.18.1/numpy-1.18.1-cp36-cp36m-manylinux1_x86_64.whl" in the cache 

 No cache entry available 

 https://<host> "GET /<downloadURL>/pypi/download/numpy/1.18.1/numpy-1.18.1-cp36-cp36m-manylinux1_x86_64.whl HTTP/1.1" 303 0 

 Status code 303 not in (200, 203, 300, 301) 

 Looking up "https://<redirectedHost>/<signedURL>" in the cache 

 No cache entry available 

 Starting new HTTPS connection (1): <redirectedHost> 

 https://<redirectedHost> "GET /<signedURL> HTTP/1.1" 200 20143300 

 Downloading https://<host>/<downloadURL>/pypi/download/numpy/1.18.1/numpy-1.18.1-cp36-cp36m-manylinux1_x86_64.whl (20.1 MB) 

 Updating cache with response from "https://<redirectedHost>/<downloadURL>" 

 Caching due to etag 

 Added numpy==1.18.1 from https://<host>/<downloadURL>/pypi/download/numpy/1.18.1/numpy-1.18.1-cp36-cp36m-manylinux1_x86_64.whl#sha256=b765ed3930b92812aa698a455847141869ef755a87e099fddd4ccf9d81fffb57 (from -r requirements.txt (line 1)) to build tracker '/tmp/pip-req-tracker-p3qwvlzb' 

 Removed numpy==1.18.1 from https://<host>/<downloadURL>/pypi/download/numpy/1.18.1/numpy-1.18.1-cp36-cp36m-manylinux1_x86_64.whl#sha256=b765ed3930b92812aa698a455847141869ef755a87e099fddd4ccf9d81fffb57 (from -r requirements.txt (line 1)) from build tracker '/tmp/pip-req-tracker-p3qwvlzb' 

Installing collected packages: numpy```

[dstufft]

pip’s caching works at the HTTP layer, so it should not cache the 303 response, but it should still be able to cache the URL that the 303 redirects to, assuming that URL is cacheable at all at the HTTP layer. It would be useful if someone could do a HEAD request against the actual file URL and see if there are any cache control headers.

[no-response[bot]]

This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don’t have enough information to take action. Please reach out if you have or find the answers we need so that we can investigate further.