pip downloads packages with VCS specifiers when using --no-index and --find-links to a local directory.

This is a duplicate of #3185, however, I’m trying to document this as a bug in a clear and actionable manner for pip’s maintainers. The bug regards pip downloading packages over the Internet rather than using locally available packages.

As mentioned in multiple places throughout the pip documentation (see here and here), using the options --no-index and --find-links </path/to/local_dir> should cause pip to install packages directly from the user’s file system, and should prevent pip from downloading and installing packages from the Internet. This behavior applies correctly to packages designated with requirements specifiers. This behavior does not apply correctly to packages designated with version control specifiers.

In the case of VCS specifiers, pip ignores the combination of --no-index and --find-links, and will instead attempt to clone and checkout the package over the Internet and install from the downloaded code. It does this regardless of whether a version of the package is present in the directory passed to --find-links.

The shell session below demonstrates the unexpected behavior.

$ pip -V
pip 7.1.2 from /Users/clasher/.virtualenvs/test/lib/python2.7/site-packages (python 2.7)
$ pwd
/Users/clasher
$ mkdir python_packages
$ pip install --download python_packages PyMySQL
Collecting PyMySQL
  Downloading PyMySQL-0.7.0-py2.py3-none-any.whl (75kB)
    100% |████████████████████████████████| 77kB 2.9MB/s
  Saved ./python_packages/PyMySQL-0.7.0-py2.py3-none-any.whl
Successfully downloaded PyMySQL
$ ls python_packages/
PyMySQL-0.7.0-py2.py3-none-any.whl
$ pip install --no-index --find-links ~/python_packages/ git+https://github.com/PyMySQL/PyMySQL.git#egg=PyMySQL
Ignoring indexes: https://pypi.python.org/simple
Collecting PyMySQL from git+https://github.com/PyMySQL/PyMySQL.git#egg=PyMySQL
  Cloning https://github.com/PyMySQL/PyMySQL.git to /private/var/folders/hl/n4818q5j4sg61mr1m2hv2yym0000gn/T/pip-build-hOBcly/PyMySQL
Installing collected packages: PyMySQL
  Running setup.py install for PyMySQL
Successfully installed PyMySQL-0.7.0

Note that pip successfully downloaded the PyMySQL wheel to ~/python_packages, but then ignored this wheel when installing PyMySQL, when specified using a VCS specifier. This is not the expected behavior.

The expected behavior is that pip will use the available downloaded wheel ~/python_packages/PyMySQL-0.7.0-py2.py3-none-any.whl. pip correctly exhibits this behavior when we provide only the package name (requirements specifier), itself:

$ pip uninstall PyMySQL
Uninstalling PyMySQL-0.7.0:
  /Users/clasher/.virtualenvs/test/lib/python2.7/site-packages/PyMySQL-0.7.0-py2.7.egg-info
  /Users/clasher/.virtualenvs/test/lib/python2.7/site-packages/pymysql/__init__.py
  […]
  /Users/clasher/.virtualenvs/test/lib/python2.7/site-packages/pymysql/util.py
  /Users/clasher/.virtualenvs/test/lib/python2.7/site-packages/pymysql/util.pyc
Proceed (y/n)? y
  Successfully uninstalled PyMySQL-0.7.0
(test)[clasher@clasher-MBP]─[26197]─[11:53]──[~]
$ pip install --no-index --find-links ~/python_packages/ PyMySQL
Ignoring indexes: https://pypi.python.org/simple
Collecting PyMySQL
Installing collected packages: PyMySQL
Successfully installed PyMySQL-0.7.0

To correct this behavior, VCS specifiers should be parsed for the package name (given in the trailing #egg=<package name>), and search for a wheel/zip/tarball matching that package name in the local directory.

[pfmoore]

Probably not. If the URL requirement had dependencies, the finder options would affect those, and I’ve no idea how you’d catch that case.

[gotgenes]

@xavfernandez, thanks for your reply. Regarding this point,

Concerning --find-links, pip has no way to know if any of the wheels in the directory corresponds to the unknown version currently hosted at git+https://github.com/PyMySQL/PyMySQL.git.

perhaps this is the real issue.

For VCS specifiers, users encode their desired package in the form of a trailing #egg=<PackageName>. In my example of, git+https://github.com/PyMySQL/PyMySQL.git#egg=PyMySQL, I specified #egg=PyMySQL. Note that I did not specify a particular version, like #egg=PyMySQL==0.7.0. In this case, I agree, pip would have a difficult time establishing a priori that the version at the Git location matches the requested version. Without the version specifier, however, as a user, I expect pip to defer to accepting whatever version is present in the directory specified by --find-links. There is no conflict of me asking pip to check the version specifier for a match.