Multiple constraints on same package cause O(exp(N)) checks

poetry init -n
poetry add -vvv numpy ipython jupyter opencv-python notebook pandas django~3.2

Runs very long dependency resolution.

Problem is that it does a lot of override attempts (why?). 0: Retrying dependency resolution with the following overrides ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}).

including weird ones like 0: Retrying dependency resolution with the following overrides ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}}).

52 override checks for this case, so, if you now have several additional slow loading packages, e.g. each override checking would take 20 seconds, you’ll get 20 * 52 = 520 seconds to install.

0: Complete version solving took 13.010 seconds with 52 overrides 0: Resolved with overrides: ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (>=3.10.0.0)>}}), ({Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.21.0)>}, Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('black', '21.12b0'): {'typing-extensions': <Dependency typing-extensions (!=3.10.0.1)>}})

In more complicated cases it will be:

0: Duplicate dependencies for numpy 0: Different requirements found for numpy (>=1.18.5) with markers platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10", numpy (>=1.19.2) with markers platform_machine == "aarch64" and python_version < "3.10" and numpy (>=1.20.0) with markers platform_machine == "arm64" and python_version < "3.10". and 0: Complete version solving took 115.009 seconds with 16 overrides 0: Resolved with overrides: ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.21.2)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.19.3)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.14.5)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.18.5)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.19.2)>}}), ({Package('opencv-python', '4.5.5.62'): {'numpy': <Dependency numpy (>=1.17.3)>}, Package('pandas', '1.4.0'): {'numpy': <Dependency numpy (>=1.20.0)>}})

[jamespacileo]

Alternative fix… keep Python to ^3.7…

and set pandas to ~1.3

Versions of Pandas of 1.4 onwards don’t support Python 3.7… and lead you into the poetry dependency resolution black hole where time ends and nothing makes sense anymore.

For I got lost in the depths of the abyss and emerged to tell the tale… shall these words spread, inform and aid other unsuspecting travellers.

The abyss is there. Shrouded. Waiting for its next victim.

[alimanfoo]

Hi all, just to share I seem to have found a way out of this abyss that allows supporting multiple Python versions from 3.7 to 3.9 at least.

In pyproject.toml I have:

[tool.poetry.dependencies]
python = ">=3.7.1,<3.10"
numpy = "*"
pandas = [
    {version="<1.4", python=">=3.7.1,<3.8"},
    {version="*", python=">=3.8,<3.10"}
]

This also only seems to work with poetry 1.2.0b3, not poetry 1.1 (which seems to omit including hashes for alternate pandas versions in the lock file).