pip 20.2.4 with 2020 resolver does not use pinned version in the constraints file

What did you want to do?

Consider this requirements.txt:

-c constraints.txt

Django==3.1 \
    --hash=sha256:1a63f5bb6ff4d7c42f62a519edc2adbb37f9b78068a5a862beff858b68e3dc8b \
    --hash=sha256:2d390268a13c655c97e0e2ede9d117007996db692c1bb93eabebd4fb7ea7012b

Django 3.1 has three dependencies, listed with hashes in constraints.txt:

pytz==2020.1 \
    --hash=sha256:a494d53b6d39c3c6e44c3bec237336e14305e4f29bbf800b599253057fbb79ed \
    --hash=sha256:c35965d010ce31b23eeb663ed3cc8c906275d6be1a34393a1d73a41febf4a048
sqlparse==0.3.1 \
    --hash=sha256:022fb9c87b524d1f7862b3037e541f68597a730a8843245c349fc93e1643dc4e \
    --hash=sha256:e162203737712307dfe78860cc56c8da8a852ab2ee33750e33aeadf38d12c548
asgiref==3.2.10 \
    --hash=sha256:7e51911ee147dd685c3c8b805c0ad0cb58d360987b56953878f8c06d2d1c6f1a \
    --hash=sha256:9fc6fb5d39b8af147ba40765234fa822b39818b12cc80b35ad9b0cef3a476aed

With pip 20.2.4, this installs the four packages:

pip install -r requirements.txt

Using the 2020 resolver fails:

pip install -r requirements.txt --use-feature=2020-resolver

It also fails with the in-development version of pip.

Output

Working output:

$ pip install -r requirements.txt 
Collecting Django==3.1
  Using cached Django-3.1-py3-none-any.whl (7.8 MB)
Collecting asgiref==3.2.10
  Using cached asgiref-3.2.10-py3-none-any.whl (19 kB)
Collecting sqlparse==0.3.1
  Using cached sqlparse-0.3.1-py2.py3-none-any.whl (40 kB)
Collecting pytz==2020.1
  Using cached pytz-2020.1-py2.py3-none-any.whl (510 kB)
Installing collected packages: pytz, sqlparse, asgiref, Django
Successfully installed Django-3.1 asgiref-3.2.10 pytz-2020.1 sqlparse-0.3.1

Failed output:

$ pip install -r requirements.txt --use-feature=2020-resolver
Collecting Django==3.1
  Using cached Django-3.1-py3-none-any.whl (7.8 MB)
Collecting asgiref~=3.2.10
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    asgiref~=3.2.10 from https://files.pythonhosted.org/packages/d5/eb/64725b25f991010307fd18a9e0c1f0e6dff2f03622fc4bcbcdb2244f60d6/asgiref-3.2.10-py3-none-any.whl#sha256=9fc6fb5d39b8af147ba40765234fa822b39818b12cc80b35ad9b0cef3a476aed (from Django==3.1->-r requirements.txt (line 3))

The dependency that causes the error can vary from call to call.

Additional information

Here’s the output of pipdeptree for the successful case:

$ pipdeptree
Django==3.1
  - asgiref [required: ~=3.2.10, installed: 3.2.10]
  - pytz [required: Any, installed: 2020.1]
  - sqlparse [required: >=0.2.2, installed: 0.3.1]
pipdeptree==1.0.0
  - pip [required: >=6.0.0, installed: 20.2.4]
setuptools==49.6.0
wheel==0.35.1

This was first reported in issue #8792, which is marked as fixed in pip 20.2.4 with PR #8839.

A working demo is at https://github.com/jwhitlock/pip-resolver-demo. I’ve simplified it to this example since it was first written.

[aawilson]

Got here from #9243. It seems weird to suddenly expect that, when a constraints file specifies a version pin and a hash, the resolver would ignore the hash completely when requirements doesn’t specify it. That seems to make constraints files essentially useless, because now for anyone using require-hashes, it makes no sense to specify the dependency in both requirements.txt and constraints.txt, and it also doesn’t make sense to go through repeated installs and find every subdep specified with a range and special-casing those for requirements.txt. Is there some reasoning someone can point to for breaking this flow?

[uranusjr]

I modified the title to reflect my observation to the issue. The error message indicates that hashes are not the issue, but the resolver not “using” the asgiref==3.2.10 line in constraints. It uses asgiref~=3.2.10 from Django instead, causing the failure.

With the current implementation, the user is expected to == all requirements as concrete requirements, and constraints don’t count. Personally I feel both pip’s current behaviour and your expectation make sense, and is (very) slightly in favour of making this work as you expect.