PIP installs wrong dependencies when installing from git branch

Environment

• pip version: 19.2.3
• Python version: 3.6
• OS: Ubuntu 18.04

Description

Installing this python project from github, from this branch: https://github.com/harvimt/quamash/tree/tests-2019

Using this line in my requirements file: git+https://github.com/harvimt/quamash.git@tests-2019

Using command pip3 install -r requirements.txt

PIP installs both pyside2 and pyqt5. These are optional dependencies and should not be installed.

Expected behavior PIP should ignore the optional dependencies of quamash, and not install pyside2 nor pyqt5.

How to Reproduce Create a new virtual environement with latest pip 19.2 and latest setuptools Create a “requirements.txt” containing:

git+https://github.com/harvimt/quamash.git@tests-2019

run pip3 install -r requirements.txt See both “pyside2” and “pyqt5” are installed.

Output

$> pip3 install -r ./requirements.txt
Collecting git+https://github.com/harvimt/quamash.git@tests-2019 (from -r ./requirements.txt (line 3))
  Cloning https://github.com/harvimt/quamash.git (to revision tests-2019) to /tmp/pip-req-build-tyfdl8pc
  Running command git clone -q https://github.com/harvimt/quamash.git /tmp/pip-req-build-tyfdl8pc
  Running command git checkout -b tests-2019 --track origin/tests-2019
  Switched to a new branch 'tests-2019'
  Branch 'tests-2019' set up to track remote branch 'tests-2019' from 'origin'.
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting pyside2<6.0,>=5.12 (from quamash==0.6.1->-r ./requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/1d/8d/591940218c923d87f48ad92d079731a373d84dcf240eb9458e3566edbf5c/PySide2-5.13.1-5.13.1-cp35.cp36.cp37-abi3-manylinux1_x86_64.whl
Collecting PyQt5<6.0,>=5.12 (from quamash==0.6.1->-r ./requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/12/63/a7a32557fdce42b1ddf2353d9e2a576143376046ead0b1918fd4cfe01b90/PyQt5-5.13.1-5.13.1-cp35.cp36.cp37.cp38-abi3-manylinux1_x86_64.whl
Requirement already satisfied: shiboken2==5.13.1 in ./venv/lib/python3.6/site-packages (from pyside2<6.0,>=5.12->quamash==0.6.1->-r ./requirements.txt (line 3)) (5.13.1)
Collecting PyQt5_sip<13,>=4.19.19 (from PyQt5<6.0,>=5.12->quamash==0.6.1->-r ./requirements.txt (line 3))
  Using cached https://files.pythonhosted.org/packages/f2/5e/da07d1977cf54d636e44a50271bc0ad1f7f34b502e484b4cefd6a1a6962a/PyQt5_sip-4.19.19-cp36-cp36m-manylinux1_x86_64.whl
Building wheels for collected packages: quamash
  Building wheel for quamash (PEP 517) ... done
  Created wheel for quamash: filename=quamash-0.6.1-cp36-none-any.whl size=13920 sha256=338dd219770fe3471d7bfb78d50f2fe747d7d383e1f8ed98de2d3f3ede6d10d0
  Stored in directory: /tmp/pip-ephem-wheel-cache-br69g3u8/wheels/fc/38/b7/8d94b7e75b8a93baeef80538e1ef17bbc25858388918a90ee2
Successfully built quamash
Installing collected packages: pyside2, PyQt5-sip, PyQt5, quamash
Successfully installed PyQt5-5.13.1 PyQt5-sip-4.19.19 pyside2-5.13.1 quamash-0.6.1

[ashleysommer]

Thank you. Closing this issue.

[chrahunt]

Hi @ashleysommer. Pip uses the PEP 517 build backend indicated in the pyproject.toml file, Poetry. python setup.py bdist_wheel uses setuptools to process the options provided in the setup.py. They are very different, so it makes sense that they would behave differently. That setup.py is generated by Poetry as a convenience so we can’t really get any conclusions from it behaving differently.

To see if pip is doing anything strange, I ran:

repro.sh

#!/bin/sh
cd "$(mktemp -d)"
git clone https://github.com/harvimt/quamash.git
cd quamash
git checkout tests-2019
pip wheel .
unzip -p quamash-0.6.1-py3-none-any.whl '*METADATA' | grep Requires-Dist

We can see:

output

Cloning into 'quamash'...
remote: Enumerating objects: 71, done.
remote: Counting objects: 100% (71/71), done.
remote: Compressing objects: 100% (38/38), done.
remote: Total 1574 (delta 32), reused 60 (delta 31), pack-reused 1503
Receiving objects: 100% (1574/1574), 59.61 MiB | 54.50 MiB/s, done.
Resolving deltas: 100% (831/831), done.
Branch 'tests-2019' set up to track remote branch 'tests-2019' from 'origin'.
Switched to a new branch 'tests-2019'
Processing /tmp/user/1000/tmp.YuThbtwdte/quamash
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting PyQt5<6.0,>=5.12 (from quamash==0.6.1)
  Using cached https://files.pythonhosted.org/packages/12/63/a7a32557fdce42b1ddf2353d9e2a576143376046ead0b1918fd4cfe01b90/PyQt5-5.13.1-5.13.1-cp35.cp36.cp37.cp38-abi3-manylinux1_x86_64.whl
  Saved ./PyQt5-5.13.1-5.13.1-cp35.cp36.cp37.cp38-abi3-manylinux1_x86_64.whl
Collecting pyside2<6.0,>=5.12 (from quamash==0.6.1)
  Using cached https://files.pythonhosted.org/packages/1d/8d/591940218c923d87f48ad92d079731a373d84dcf240eb9458e3566edbf5c/PySide2-5.13.1-5.13.1-cp35.cp36.cp37-abi3-manylinux1_x86_64.whl
  Saved ./PySide2-5.13.1-5.13.1-cp35.cp36.cp37-abi3-manylinux1_x86_64.whl
Collecting PyQt5_sip<13,>=4.19.19 (from PyQt5<6.0,>=5.12->quamash==0.6.1)
  Using cached https://files.pythonhosted.org/packages/bf/63/d363b567237375235d270049fd6416c83949f470bb51539458966cd7a38c/PyQt5_sip-4.19.19-cp37-cp37m-manylinux1_x86_64.whl
  Saved ./PyQt5_sip-4.19.19-cp37-cp37m-manylinux1_x86_64.whl
Collecting shiboken2==5.13.1 (from pyside2<6.0,>=5.12->quamash==0.6.1)
  Using cached https://files.pythonhosted.org/packages/00/cc/76b670f0efbbcdac08b5707f021bb0647c6449196857e043af7edbc521db/shiboken2-5.13.1-5.13.1-cp35.cp36.cp37-abi3-manylinux1_x86_64.whl
  Saved ./shiboken2-5.13.1-5.13.1-cp35.cp36.cp37-abi3-manylinux1_x86_64.whl
Skipping PyQt5, due to already being wheel.
Skipping pyside2, due to already being wheel.
Skipping PyQt5-sip, due to already being wheel.
Skipping shiboken2, due to already being wheel.
Building wheels for collected packages: quamash
  Building wheel for quamash (PEP 517) ... done
  Created wheel for quamash: filename=quamash-0.6.1-py3-none-any.whl size=13920 sha256=338dd219770fe3471d7bfb78d50f2fe747d7d383e1f8ed98de2d3f3ede6d10d0
  Stored in directory: /tmp/user/1000/tmp.YuThbtwdte/quamash
Successfully built quamash
Requires-Dist: PyQt5 (>=5.12,<6.0)
Requires-Dist: pyside2 (>=5.12,<6.0)

The important part is

Requires-Dist: PyQt5 (>=5.12,<6.0)
Requires-Dist: pyside2 (>=5.12,<6.0)

So it looks like it’s Poetry returning these values when being used to build the wheel. If that is unexpected I would raise an issue on their issue tracker. One hint, if it helps, is that the Poetry documentation seems to indicate that optional dependencies are meant to be used in extras. When creating a dummy extra in the pyproject.toml like

[tool.poetry.extras]
foo = ["pyside2", "PyQt5"]

then the generated wheel has a different Requires-Dist:

Requires-Dist: PyQt5 (>=5.12,<6.0); extra == "foo"
Requires-Dist: pyside2 (>=5.12,<6.0); extra == "foo"

so it may be that Poetry is just putting these with no suffix in the case where they are not associated with any extra.