Fast-deps: bad interaction with Artifactory leading to zipfile.BadZipFile
See original GitHub issueWhat did you want to do? Trying to install Cython using the experimental features.
Command line:
virtualenv -p python3.7 env
source env/bin/activate
pip install -U pip
pip install --use-feature=2020-resolver --use-feature=fast-deps -v cython
This works correctly on my home linux computer, but fails at work where we have the following /etc/pip.conf:
[global]
cert = /etc/pki/tls/cert.pem
no-cache-dir = false
index-url = https://artifactory.our-organization.com/artifactory/api/pypi/python-virtual-repo/simple
index = https://artifactory.our-organization.com/artifactory/api/pypi/python-virtual-repo/simple
It also works in both places when I keep 2020-resolver but omit fast-deps.
The same problem is present on master
Output This is the relevant part of the output from pip -v, with pip f17c1d6d3bf4b4ee8b7b0e669c449b9cd86e94f6 installed:
Given no hashes to check 141 links for project 'cython': discarding no candidates
Collecting cython
  Obtaining dependency information from cython 0.29.21
  https://artifactory.our-organization.com:443 "HEAD /artifactory/api/pypi/python-virtual-repo/packages/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 200 0
  https://artifactory.our-organization.com:443 "GET /artifactory/api/pypi/python-virtual-repo/packages/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 200 1969113
  https://artifactory.our-organization.com:443 "GET /artifactory/api/pypi/python-virtual-repo/packages/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 200 1969113
  https://artifactory.our-organization.com:443 "GET /artifactory/api/pypi/python-virtual-repo/packages/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 200 1969113
  https://artifactory.our-organization.com:443 "GET /artifactory/api/pypi/python-virtual-repo/packages/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 200 1969113
  https://artifactory.our-organization.com:443 "GET /artifactory/api/pypi/python-virtual-repo/packages/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 200 1969113
  https://artifactory.our-organization.com:443 "GET /artifactory/api/pypi/python-virtual-repo/packages/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 200 1969113
  https://artifactory.our-organization.com:443 "GET /artifactory/api/pypi/python-virtual-repo/packages/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 200 1969113
[SNIP: that line repeats a total of 193 times]
ERROR: Exception:
Traceback (most recent call last):
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 216, in _main
    status = self.run(options, args)
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper
    return func(self, options, args)
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 324, in run
    reqs, check_supported_wheels=not options.target_dir
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 111, in resolve
    requirements, max_rounds=try_to_avoid_resolution_too_deep,
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 427, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 324, in resolve
    failure_causes = self._attempt_to_pin_criterion(name, criterion)
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 224, in _attempt_to_pin_criterion
    criteria = self._get_criteria_to_update(candidate)
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_vendor/resolvelib/resolvers.py", line 215, in _get_criteria_to_update
    for r in self._p.get_dependencies(candidate):
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/provider.py", line 151, in get_dependencies
    for r in candidate.iter_dependencies(with_requires)
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/provider.py", line 150, in <listcomp>
    r
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 251, in iter_dependencies
    for r in self.dist.requires():
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 230, in dist
    self._prepare()
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 217, in _prepare
    dist = self._prepare_distribution()
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 315, in _prepare_distribution
    self._ireq, parallel_builds=True,
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 500, in prepare_linked_requirement
    wheel_dist = self._fetch_metadata_using_lazy_wheel(link)
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 488, in _fetch_metadata_using_lazy_wheel
    return dist_from_wheel_url(name, url, self._session)
  File "/tmp/tmp.O9hMH57Ecq/env2/lib/python3.7/site-packages/pip/_internal/network/lazy_wheel.py", line 46, in dist_from_wheel_url
    zip_file = ZipFile(wheel)  # type: ignore
  File "/opt/python-3.7/lib/python3.7/zipfile.py", line 1222, in __init__
    self._RealGetContents()
  File "/opt/python-3.7/lib/python3.7/zipfile.py", line 1317, in _RealGetContents
    raise BadZipFile("Bad magic number for central directory")
zipfile.BadZipFile: Bad magic number for central directory
The same part of the output for my home machine where it works, also with f17c1d6d3bf4b4ee8b7b0e669c449b9cd86e94f6
Given no hashes to check 141 links for project 'cython': discarding no candidates
Collecting cython
  Obtaining dependency information from cython 0.29.21
  Starting new HTTPS connection (1): files.pythonhosted.org:443
  https://files.pythonhosted.org:443 "HEAD /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 200 0
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 3033
  Status code 206 not in (200, 203, 300, 301)
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 7207
  Status code 206 not in (200, 203, 300, 301)
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 13941
  Status code 206 not in (200, 203, 300, 301)
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 166
  Status code 206 not in (200, 203, 300, 301)
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 10240
  Status code 206 not in (200, 203, 300, 301)
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 30
  Status code 206 not in (200, 203, 300, 301)
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 41
  Status code 206 not in (200, 203, 300, 301)
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 99
  Status code 206 not in (200, 203, 300, 301)
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 30
  Status code 206 not in (200, 203, 300, 301)
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 33
  Status code 206 not in (200, 203, 300, 301)
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 1035
  Status code 206 not in (200, 203, 300, 301)
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 30
  Status code 206 not in (200, 203, 300, 301)
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 38
  Status code 206 not in (200, 203, 300, 301)
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 21
  Status code 206 not in (200, 203, 300, 301)
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 30
  Status code 206 not in (200, 203, 300, 301)
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  Request header has "no-cache", cache bypassed
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 206 3404
  Status code 206 not in (200, 203, 300, 301)
  Created temporary directory: /tmp/pip-unpack-xywl9x_9
  Looking up "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl" in the cache
  No cache entry available
  https://files.pythonhosted.org:443 "GET /packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl HTTP/1.1" 200 1969113
  Downloading Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl (2.0 MB)
     |ββββββββββββββββββββββββββββββββ| 2.0 MB 2.2 MB/s eta 0:00:01  Ignoring unknown cache-control directive: immutable
  Updating cache with response from "https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl"
  Caching due to etag
     |ββββββββββββββββββββββββββββββββ| 2.0 MB 2.2 MB/s 
  Added cython from https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl#sha256=5e545a48f919e40079b0efe7b0e081c74b96f9ef25b9c1ff4cdbd95764426b58 to build tracker '/tmp/pip-req-tracker-a7imwe9c'
  Removed cython from https://files.pythonhosted.org/packages/6b/36/d6c18632a339dafa54fd128b0dd2c36c6dc4bc86b8e0d366ccd9f22b480a/Cython-0.29.21-cp37-cp37m-manylinux1_x86_64.whl#sha256=5e545a48f919e40079b0efe7b0e081c74b96f9ef25b9c1ff4cdbd95764426b58 from build tracker '/tmp/pip-req-tracker-a7imwe9c'
Installing collected packages: cython
  changing mode of /tmp/ohoashd-env/bin/cygdb to 755
  changing mode of /tmp/ohoashd-env/bin/cython to 755
  changing mode of /tmp/ohoashd-env/bin/cythonize to 755
Successfully installed cython-0.29.21
Removed build tracker: '/tmp/pip-req-tracker-a7imwe9c'
Additional information
Happy to try versions from pull requests if thatβs helpful. I might also be smart enough to set up a mitmproxy to record the HTTP interaction, but Iβm not completely sure.
Issue Analytics
- State:
 - Created 3 years ago
 - Comments:27 (25 by maintainers)
 
Top Results From Across the Web
pip - Fast-deps: Artifactoryμμ μλͺ»λ μνΈ μμ©μΌλ‘ μΈν΄ zipfile ...
Pip: Fast-deps: Artifactoryμμ μλͺ»λ μνΈ μμ©μΌλ‘ μΈν΄ zipfile. ... _RealGetContents raise BadZipFile("Bad magic number for central directory") zipfile.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found

Howdy, looks like a bug was filed against warehouse (PyPI) regarding this.
I wanted to drop in to note that the CDN for PyPI files (files.pythonhosted.org) has continuously supported Range requests since its inception.
The confusion seems to be around some copy pasta
curlcommands being used to test the functionality.Our CDN does not respond perfectly to Range requests when they come in as a HEAD request, but does for valid GET requests with a Range header.
Note that many of the examples in this thread include the
-Iparameter tocurlleading to a HEAD request.Because our CDN fetches the full file to cache at edge before responding to Range GET requests, the HEAD is the response for the full object, always.
I would certainly hope removing old resolver would be pushed out into horizon until one or more of these speedups is in place in both Warehouse and downstream mirrors. That may take years for server features.