Wheels and PEP440 resolver

Description

In one of my projects CI, we build the projects and all its dependencies into wheels then install all these in a docker container:

python3 setup.py bdist_wheel
pip3 wheel -w dist/ .
[...]
pip3 install dist/*

The project has an internal dependency, referenced using PEP440 style to be able to specify the origin AND the version: install_requires=['utilities @ https://xxx.com/utilities-3.6.0-py3-none-any.whl']

When doing the installation, the resolver fails, with the following logs:

ERROR: Cannot install cloud==3.2.12 and utilities 3.6.0 (from /home/olemoign/dist/utilities-3.6.0-py3-none-any.whl) because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested utilities 3.6.0 (from /home/olemoign/dist/utilities-3.6.0-py3-none-any.whl)
    cloud 3.2.12 depends on utilities 3.6.0 (from https://xxx.com/utilities-3.6.0-py3-none-any.whl)

The resolver seems to be unable to reconcile that it’s the same version. Is there any way to solve this?

Expected behavior

Functional installation.

pip version

21.1.1

Python version

3.8.9

OS

Ubuntu 21.04

How to Reproduce

Full description posted above. I can create github repos if needed.

Output

ERROR: Cannot install cloud==3.2.12 and utilities 3.6.0 (from /home/olemoign/dist/utilities-3.6.0-py3-none-any.whl) because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested utilities 3.6.0 (from /home/olemoign/dist/utilities-3.6.0-py3-none-any.whl)
    cloud 3.2.12 depends on utilities 3.6.0 (from https://xxx.com/utilities-3.6.0-py3-none-any.whl)

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pfmoore]

My surprise stems from the fact that it works with the old resolver --use-deprecated=legacy-resolver.

That’s because the old resolver ignored the conflict, not because it wasn’t there…

I’m surprised that I’m the only one feeling that index-url/extra-index-url is bad.

You’re not. There’s been a lot of publicity over “dependency confusion” attacks, which are basically the problem you have (private name shadowed by a project on PyPI). There’s work going on to look for solutions at the PyPI level (around reserving namespaces) and we’re looking at responses from pip’s side (not necessarily fixes, more ways to help people not accidentally stumble into the problem, which doesn’t help you because you’re aware of the issue).

For your situation, though, the best resolution is probably to simply remove utilities-3.6.0-py3-none-any.whl from your dist directory so you get it via the direct URL.

[olemoign]

Thank you very much for your detailed answer. Much appreciated.