new dependency resolver runs "forever" due to incompatible package versions

The new resolver in pip iterates over all versions of all packages, even if a small number (e.g. 2) of the specifications are completely incompatible.

Tested with latest pip from master.

What did you want to do? Something like (this is just a toy example; using different packages for our application): pip install --use-feature=2020-resolver marshmallow-sqlalchemy sqlalchemy==0.6.0 psycopg2-binary scipy

All available versions of marshmallow-sqlalchemy require sqlalchemy of at least 0.7, so the first two specifications are incompatible and will reasonably quickly (~11 seconds on my machine) give a failure if specified by themselves. However, adding additional packages with un-pinned versions (e.g. scipy, which depends on numpy) introduces an exponentially increasing number of options. The new dependency resolver apparently feels the need to test all of these, even though it could reasonably fail after discovering that the first two specifications were incompatible. I didn’t let it run to completion, but I suspect it would literally take days.

Output

# time pip install --use-feature=2020-resolver marshmallow-sqlalchemy sqlalchemy==0.6.0 psycopg2-binary scipy
Collecting sqlalchemy==0.6.0
  Using cached SQLAlchemy-0.6.0.tar.gz (1.8 MB)
Collecting psycopg2-binary
  Using cached psycopg2_binary-2.8.6-cp36-cp36m-manylinux1_x86_64.whl (3.0 MB)
Collecting scipy
  Using cached scipy-1.5.2-cp36-cp36m-manylinux1_x86_64.whl (25.9 MB)
Collecting numpy>=1.14.5
  Using cached numpy-1.19.2-cp36-cp36m-manylinux2010_x86_64.whl (14.5 MB)
Collecting marshmallow-sqlalchemy
  Using cached marshmallow_sqlalchemy-0.23.1-py2.py3-none-any.whl (18 kB)
  Using cached marshmallow_sqlalchemy-0.23.0-py2.py3-none-any.whl (18 kB)
  Using cached marshmallow_sqlalchemy-0.22.3-py2.py3-none-any.whl (18 kB)
  Using cached marshmallow_sqlalchemy-0.22.2-py2.py3-none-any.whl (18 kB)
  Using cached marshmallow_sqlalchemy-0.22.1-py2.py3-none-any.whl (18 kB)
  Using cached marshmallow_sqlalchemy-0.22.0-py2.py3-none-any.whl (18 kB)
  Using cached marshmallow_sqlalchemy-0.21.0-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.20.0-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.19.0-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.18.0-py2.py3-none-any.whl (13 kB)
  Using cached marshmallow_sqlalchemy-0.17.2-py2.py3-none-any.whl (13 kB)
  Using cached marshmallow_sqlalchemy-0.17.1-py2.py3-none-any.whl (13 kB)
  Using cached marshmallow_sqlalchemy-0.17.0-py2.py3-none-any.whl (13 kB)
  Using cached marshmallow_sqlalchemy-0.16.4-py2.py3-none-any.whl (13 kB)
  Using cached marshmallow_sqlalchemy-0.16.3-py2.py3-none-any.whl (13 kB)
  Using cached marshmallow_sqlalchemy-0.16.2-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.16.1-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.16.0-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.15.0-py2.py3-none-any.whl (11 kB)
  Using cached marshmallow_sqlalchemy-0.14.2-py2.py3-none-any.whl (11 kB)
  Using cached marshmallow_sqlalchemy-0.14.1-py2.py3-none-any.whl (9.6 kB)
  Using cached marshmallow_sqlalchemy-0.14.0-py2.py3-none-any.whl (9.6 kB)
  Using cached marshmallow_sqlalchemy-0.13.2-py2.py3-none-any.whl (11 kB)
  Using cached marshmallow_sqlalchemy-0.13.1-py2.py3-none-any.whl (11 kB)
  Using cached marshmallow_sqlalchemy-0.13.0-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.12.1-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.12.0-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.11.0-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.10.0-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.9.0-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.8.1-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.8.0-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.7.1-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.7.0-py2.py3-none-any.whl (12 kB)
  Using cached marshmallow_sqlalchemy-0.6.0-py2.py3-none-any.whl (11 kB)
  Using cached marshmallow_sqlalchemy-0.5.0-py2.py3-none-any.whl (11 kB)
  Using cached marshmallow_sqlalchemy-0.4.1-py2.py3-none-any.whl (11 kB)
  Using cached marshmallow_sqlalchemy-0.4.0-py2.py3-none-any.whl (11 kB)
  Using cached marshmallow_sqlalchemy-0.3.0-py2.py3-none-any.whl (10 kB)
  Using cached marshmallow_sqlalchemy-0.2.0-py2.py3-none-any.whl (10 kB)
  Using cached marshmallow_sqlalchemy-0.1.1-py2.py3-none-any.whl (10 kB)
  Using cached marshmallow_sqlalchemy-0.1.0-py2.py3-none-any.whl (10 kB)
Collecting numpy>=1.14.5
  Using cached numpy-1.19.1-cp36-cp36m-manylinux2010_x86_64.whl (14.5 MB)
  Using cached numpy-1.19.0-cp36-cp36m-manylinux2010_x86_64.whl (14.6 MB)
  Downloading numpy-1.18.5-cp36-cp36m-manylinux1_x86_64.whl (20.1 MB)
     |████████████████████████████████| 20.1 MB 8.7 MB/s 
  Downloading numpy-1.18.4-cp36-cp36m-manylinux1_x86_64.whl (20.2 MB)
     |████████████████████████████████| 20.2 MB 3.3 MB/s 
  Downloading numpy-1.18.3-cp36-cp36m-manylinux1_x86_64.whl (20.2 MB)
     |████████████████████████████████| 20.2 MB 3.3 MB/s 
  Downloading numpy-1.18.2-cp36-cp36m-manylinux1_x86_64.whl (20.2 MB)
     |████████████████████████████████| 20.2 MB 4.6 MB/s 
^CERROR: Operation cancelled by user

real	1m25.119s
user	1m2.579s
sys	0m3.083s

Additional information

We had something similar to this in our application (pinned version of sqlalchemy, unpinned version of another package that depended on it and had ~200 published versions), and it was literally at a point of running for 45 minutes with no output to the console (verbose mode did help). I ended up having to debug by iteratively removing things until I got down to the incompatible packages.

[uranusjr]

You hit the nail right on the head here 🙂

I think one simple way pip can do here is to ask for help eagerly, even if it hasn’t exhaused all possibilities. It is very rarely the user actually wants marshmallow-sqlalchemy 0.1.0 when they specify a large version rage, and pip can stops after trying a few times and just aborts saying “hey eh I haven’t tried everything but marshmallow-sqlalchemy is taking too much time, please tell me more especifically what you need”. This goes back to the ResolutionTooDeep discussion in #8683; what would be a good metric to abort here? Times the resolver backtracked? Or maybe times the resolver backtracked for a specific package?

[brainwane]

@bkurtz Thank you for the issue report! In #8683, #8346, #8975, and some related issues and pull requests, we addressed the fundamental problem of how much and in what cases pip backtracks, and a user-visible part of that which is the informative and error messages pip outputs during backtracking.

In today’s team meeting we decided that this is sufficient to close this issue. @pradyunsg will file a followup issue about the idea of printing package names better in error output.

Thanks @bkurtz and I hope you can also test the pip 20.3 beta release in case there are other issues for you to report!