The new dependency resolver does not handle impossible dependency configurations well

Description

The dependency resolver in pip 21.1.1 takes a really long time (couldn’t finish within 25 mins in CI pipeline) with an impossible dependency configuration. The legacy resolver in pip 20.3.4 could resolve it in seconds.

Expected behavior

The new resolver should be able to detect impossible configurations more efficiently (not sure how long it would have taken).

pip version

21.1.1

Python version

3.7

OS

Any

How to Reproduce

Below is an example with a requirements.txt that can be used to reproduce the setting. We had one internal package requiring numpy, but the conflict would have occurred without it as well. I have included the important details about the private package here in case it makes a difference.

# requirements.txt

# Private wheel that depends on numpy>=1.16.1
private-wheel~=0.1.5

# Public packages
h5py~=2.10.0
numpy==1.16.6
opencv-python==3.4.13.47
scipy==1.4.1
tensorflow~=2.4.1

Install with (we had our private-wheel in the wheels directory):

python -m pip install -f wheels -r requirements.txt

pip 20.3.4 provides the following explanation within seconds:

INFO: pip is looking at multiple versions of numpy to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install -r requirements.txt (line 15), -r requirements.txt (line 17), -r requirements.txt (line 5), -r requirements.txt (line 8), envision-engine==0.1.5 and numpy==1.16.6 because these package versions have conflicting dependencies.
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

The conflict is caused by:
    The user requested numpy==1.16.6
    opencv-python 3.4.13.47 depends on numpy>=1.14.5
    scipy 1.4.1 depends on numpy>=1.13.3
    private-wheel 0.1.5 depends on numpy>=1.16.1
    h5py 2.10.0 depends on numpy>=1.7
    tensorflow 2.4.1 depends on numpy~=1.19.2

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

WARNING: You are using pip version 20.3.4; however, version 21.1.1 is available.
You should consider upgrading via the '/opt/hostedtoolcache/Python/3.7.10/x64/bin/python -m pip install --upgrade pip' command

pip 21.1.1 stalled in the following and the job was cancelled after waiting over 25 min for the dependency to be resolved:

INFO: pip is looking at multiple versions of h5py to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of private-wheel to determine which version is compatible with other requirements. This could take a while.
WARNING: Location 'wheels' is ignored: it is either a non-existing path or lacks a specific scheme.

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[jlintusaari]

Here is a toy private-wheel that serves the purpose:

# setup.py

from setuptools import setup

setup(
    name='private-wheel',
    version='0.1.5',
    description='Private wheel requiring numpy',
    author='',
    author_email='',
    install_requires=['numpy>=1.16.1']
)

Place the setup.py to an empty directory and then from the directory you can build the wheel with

pip install wheel
pip wheel --no-deps .

[no-response[bot]]

This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don’t have enough information to take action. Please reach out if you have or find the answers we need so that we can investigate further.