Conflicting dependencies culprit is not displayed

Description

I have a project with two packages. I upgraded their dependencies on databind.core and databind.json from >=0.8.0,<=1.0.0 to >=1.1.0,<2.0.0. I.e. in the setup.py of both packages, the install reqs are listed as

  'databind.json >=1.1.0,<2.0.0',
  'databind.core >=1.1.0,<2.0.0',

Now installing the packages again in develop mode gives me:

$ pip install -v -e spam-bot/ -e spam-bot-predicate-comments/
[ ... ]

The conflict is caused by:
    spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.2.0 depends on databind.core<2.0.0 and >=1.2.0
    spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.1.6 depends on databind.core<2.0.0 and >=1.1.6
    spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.1.5 depends on databind.core<2.0.0 and >=1.1.5
    spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.1.4 depends on databind.core<2.0.0 and >=1.1.4
    spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.1.3 depends on databind.core<2.0.0 and >=1.1.3
    spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.1.2 depends on databind.core<2.0.0 and >=1.1.2
    spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0
    databind-json 1.1.1 depends on databind.core<2.0.0 and >=1.1.1

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

(Full logs: pip_install_logs_21_1_3.txt, pip_install_logs_21_2_4.txt)

But I don’t see how this is a conflict.

• spam-bot 0.4.2 depends on databind.core<2.0.0 and >=1.1.0 *spam-bot-predicate-comments 0.4.2 depends on databind.core<2.0.0 and >=1.1.0

Ok they both depend on databind.core with the same version cosntraint.

• databind-json 1.2.0 depends on databind.core<2.0.0 and >=1.2.0

The version constraint of databind.json is slightly stricter, but would all three constraints are satisfiable with databind.core 1.2.0.

Now it turns out that thsi is an issue because there is also a requirement on databind.yaml >=0.1.2,<1.0.0 in the setup files which I did not notice earlier. That package is not compatible with databind.json or databind.core in those versions. Removing this package as a requirement resolves the conflict.

However the confusing this is that databind.yaml does not actually show up in the conflict error message at all.

Expected behavior

Pip is able to install the dependencies, in this case in version 1.2.0 for both databind.core and databind.json (latest on PyPI).

pip version

21.1.3, 21.2.4

Python version

3.9.6

OS

OSX

How to Reproduce

1. Install a package with the below setup file:

   import setuptools
   setuptools.setup(
     name='pkgb',
     version='0.0.0',
     install_requires=[
       'databind.json >=0.9.0,<1.0.0',
       'databind.core >=0.8.0,<1.0.0',
       'databind.yaml >=0.1.2,<1.0.0'
     ]
   )
   

    $ pip install .
   

2. Update the requirements for databind.core and databind.json and install again.

   import setuptools
   setuptools.setup(
     name='pkgb',
     version='0.0.0',
     install_requires=[
       'databind.json >=1.1.0,<2.0.0',
       'databind.core >=1.1.0,<2.0.0',
       'databind.yaml >=0.1.2,<1.0.0'
     ]
   )
   

    $ pip install .
   

3. Notice the conflict error but that it doesn’t show the conflict with databind.yaml at all.

   ERROR: Cannot install pkgb and pkgb==0.0.0 because these package versions have conflicting dependencies.
   
   The conflict is caused by:
       pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
       databind-json 1.2.0 depends on databind.core<2.0.0 and >=1.2.0
       pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
       databind-json 1.1.6 depends on databind.core<2.0.0 and >=1.1.6
       pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
       databind-json 1.1.5 depends on databind.core<2.0.0 and >=1.1.5
       pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
       databind-json 1.1.4 depends on databind.core<2.0.0 and >=1.1.4
       pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
       databind-json 1.1.3 depends on databind.core<2.0.0 and >=1.1.3
       pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
       databind-json 1.1.2 depends on databind.core<2.0.0 and >=1.1.2
       pkgb 0.0.0 depends on databind.core<2.0.0 and >=1.1.0
       databind-json 1.1.1 depends on databind.core<2.0.0 and >=1.1.1
   
   To fix this you could try to:
   1. loosen the range of package versions you've specified
   2. remove package versions to allow pip attempt to solve the dependency conflict
   

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[SVAIRobotics]

I second @NiklasRosenstein and would like to report the same issue. It would be great to have this issue escalated further in terms of urgency since it’s been ~4 months since he opened this.

pip version 21.3.1

Python version 3.8.5

OS Amazon Linux 2 (on AWS EC2 / Elastic Beanstalk)

Command run and Conflict reported by pip

I ran python3 -m pip install https://github.com/django-ai/djaiot/archive/dev.zip https://github.com/fastaiapi/fastaiapi/archive/dev.zip

The conflict reported by pip varies from run to run, but is of one same head-scratching pattern as follow:

The conflict is caused by: fastaiapi 0.0.0.dev0 depends on Django-MySQL<5.0.0 and >=4.3.0 djai 0.0.0.dev0 depends on Django-MySQL<5.0.0 and >=4.4.0

The conflict is caused by: fastaiapi 0.0.0.dev0 depends on NumPy<2.0.0 and >=1.20.3 djai 0.0.0.dev0 depends on NumPy<2.0.0 and >=1.21.5

As @NiklasRosenstein mentioned above, such different, but not conflicting, dependencies should not be problematic.

The real problem (which I deliberately constructed) which is not reported by pip is actually the following:

(ideal, correct conflict report) The conflict is caused by: fastaiapi 0.0.0.dev0 depends on HuggingFace-Hub<0.3.0 and >=0.2.1 djai 0.0.0.dev0 depends on HuggingFace-Hub<0.5.0 and >=0.4.0

[notatallshaw]

Can someone please run with PIP_RESOLVER_DEBUG=1 in the environment, and post the output of a couple of runs as GitHub Gists?

Here are 3 runs: https://gist.github.com/notatallshaw/807eae4ac1735da1f19a90e17922d346

All 3 are truncated to the first 5000 lines, as no matter how many runs I did none of the logs ended up being small enough to upload to gist (varied from ~18 MBs to > 1 GB). However the first difference between the resolution step was always in the first 100 lines, usually right after Reporter.starting_round(2) but sometimes as late as right after Reporter.starting_round(4).