Resolution conflict due to the PEP 508 requirement with superfluous #egg fragment specified in a package’s metadata

Description

A conflict occurs when trying to install packages based on the requirements.txt file which has a git-based package and another package that has the same git-based package as a dependency. It happens even if there are no git revisions specified.

What’s more, it means that if a git-based package with a git-based dependency is installed, then the pip freeze output is unusable because of this conflict. The example of such a package is https://github.com/QuickLogic-Corp/quicklogic-fasm with its install_requires:

    install_requires = [
        'fasm @ git+https://github.com/symbiflow/fasm#egg=fasm',
        'fasm-utils @ git+https://github.com/QuickLogic-Corp/quicklogic-fasm-utils#egg=fasm-utils'
    ],

With such a package installed, the output of pip freeze can’t be used successfully in pip install -r.

Expected behavior

Packages should be successfully installed.

The conflict is only reasonable if git revisions are specified for the same git-based package and are different. For example, if after resolving dependencies it turns out such packages are going to be installed:

fasm-utils @ git+https://github.com/QuickLogic-Corp/quicklogic-fasm-utils@REVISION
fasm-utils @ git+https://github.com/QuickLogic-Corp/quicklogic-fasm-utils@ANOTHER_REVISION

but if only one specifies a revision it should be compatible with a package without a revision specified.

pip version

21.1.1

Python version

3.7 / 3.9

OS

Linux

How to Reproduce

python3 -m pip install git+https://github.com/QuickLogic-Corp/quicklogic-fasm
python3 -m pip freeze >requirements.txt
python3 -m pip uninstall quicklogic-fasm fasm fasm-utils
python3 -m pip install -r requirements.txt

but also without git revisions it doesn’t work:

python3 -m pip install git+https://github.com/QuickLogic-Corp/quicklogic-fasm
python3 -m pip freeze | sed 's#@[^ ]\+##g' >requirements-without-revisions.txt
python3 -m pip uninstall quicklogic-fasm fasm fasm-utils
pip install -r requirements-without-revisions.txt

Output

root@5a910819b958:/# pip install -r requirements.txt 
Collecting fasm@ git+https://github.com/symbiflow/fasm@99f199f9e32fd30c8adffcc73c13caf95a951c35
  Cloning https://github.com/symbiflow/fasm (to revision 99f199f9e32fd30c8adffcc73c13caf95a951c35) to /tmp/pip-install-31i9zc00/fasm_389620f524324d759e68423b3f833b59
  Running command git clone -q https://github.com/symbiflow/fasm /tmp/pip-install-31i9zc00/fasm_389620f524324d759e68423b3f833b59
  Running command git rev-parse -q --verify 'sha^99f199f9e32fd30c8adffcc73c13caf95a951c35'
  Running command git fetch -q https://github.com/symbiflow/fasm 99f199f9e32fd30c8adffcc73c13caf95a951c35
  Running command git submodule update --init --recursive -q
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting fasm-utils@ git+https://github.com/QuickLogic-Corp/quicklogic-fasm-utils@3d6a375ddb6b55aaa5a59d99e44a207d4c18709f
  Cloning https://github.com/QuickLogic-Corp/quicklogic-fasm-utils (to revision 3d6a375ddb6b55aaa5a59d99e44a207d4c18709f) to /tmp/pip-install-31i9zc00/fasm-utils_77d8f9716d15454fbb4dec7204d8caaa
  Running command git clone -q https://github.com/QuickLogic-Corp/quicklogic-fasm-utils /tmp/pip-install-31i9zc00/fasm-utils_77d8f9716d15454fbb4dec7204d8caaa
  Running command git rev-parse -q --verify 'sha^3d6a375ddb6b55aaa5a59d99e44a207d4c18709f'
  Running command git fetch -q https://github.com/QuickLogic-Corp/quicklogic-fasm-utils 3d6a375ddb6b55aaa5a59d99e44a207d4c18709f
Collecting quicklogic-fasm@ git+https://github.com/QuickLogic-Corp/quicklogic-fasm@b86a3f926c84f96e32e6dc51e0679a42d0f98063
  Cloning https://github.com/QuickLogic-Corp/quicklogic-fasm (to revision b86a3f926c84f96e32e6dc51e0679a42d0f98063) to /tmp/pip-install-31i9zc00/quicklogic-fasm_907a7665943043c0909a3200aee79f10
  Running command git clone -q https://github.com/QuickLogic-Corp/quicklogic-fasm /tmp/pip-install-31i9zc00/quicklogic-fasm_907a7665943043c0909a3200aee79f10
  Running command git rev-parse -q --verify 'sha^b86a3f926c84f96e32e6dc51e0679a42d0f98063'
  Running command git fetch -q https://github.com/QuickLogic-Corp/quicklogic-fasm b86a3f926c84f96e32e6dc51e0679a42d0f98063
  Running command git submodule update --init --recursive -q
Requirement already satisfied: Arpeggio==1.10.2 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 1)) (1.10.2)
Requirement already satisfied: textX==2.3.0 in /usr/local/lib/python3.9/dist-packages (from -r requirements.txt (line 5)) (2.3.0)
Collecting fasm-utils@ git+https://github.com/QuickLogic-Corp/quicklogic-fasm-utils#egg=fasm-utils
  Cloning https://github.com/QuickLogic-Corp/quicklogic-fasm-utils to /tmp/pip-install-31i9zc00/fasm-utils_ad67eecd460a46f2878c8cb548170041
  Running command git clone -q https://github.com/QuickLogic-Corp/quicklogic-fasm-utils /tmp/pip-install-31i9zc00/fasm-utils_ad67eecd460a46f2878c8cb548170041
Collecting fasm@ git+https://github.com/symbiflow/fasm#egg=fasm
  Cloning https://github.com/symbiflow/fasm to /tmp/pip-install-31i9zc00/fasm_284e261cc2924e599dcf7ec899bef64e
  Running command git clone -q https://github.com/symbiflow/fasm /tmp/pip-install-31i9zc00/fasm_284e261cc2924e599dcf7ec899bef64e
  Running command git submodule update --init --recursive -q
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
INFO: pip is looking at multiple versions of fasm-utils to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of fasm to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install -r requirements.txt (line 4) and fasm-utils 0.0.1 (from git+https://github.com/QuickLogic-Corp/quicklogic-fasm-utils@3d6a375ddb6b55aaa5a59d99e44a207d4c18709f) because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested fasm-utils 0.0.1 (from git+https://github.com/QuickLogic-Corp/quicklogic-fasm-utils@3d6a375ddb6b55aaa5a59d99e44a207d4c18709f)
    quicklogic-fasm 0.0.1 depends on fasm-utils 0.0.1 (from git+https://github.com/QuickLogic-Corp/quicklogic-fasm-utils#egg=fasm-utils)

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies



Without git revisions in `requirements.txt` the conflict information makes even less sense:

The conflict is caused by:
    The user requested fasm-utils 0.0.1 (from git+https://github.com/QuickLogic-Corp/quicklogic-fasm-utils)
    quicklogic-fasm 0.0.1 depends on fasm-utils 0.0.1 (from git+https://github.com/QuickLogic-Corp/quicklogic-fasm-utils#egg=fasm-utils)

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[uranusjr]

Thanks for the clarification. To be honest, yes, the conflict from different egg= values is condiered correct, since it is the standard compliant implementation in pip maintianers’ views. But we also understand it is counter-intuitive and have been discussing the possibility of loosening the check (practicality beats purity and things like that). I have a work-in-progress locally that implements a loosened check, and will try to push it this weekend. It is still far from ready, but I think it may be possible to get it ready for 21.2 (due next month) if I put enough time on it.

[uranusjr]

I also changed the labels. Further discussions should go to either the PR #10079 (which ignores #egg unconditionally so resolves the symptom) or #1289 (which is broader in scope and deals with the root cause).