Resolver preference improvements to avoid trying a dependency for too many times

setuptools version

setuptools==52.0.0

Python version

Python 3.8

OS

MacOS 11.2.3

Additional environment information

I only tried to reproduce this inside a venv. I also tried with setuptools==57.2.0, as well as Python 3.9 and ran into the same issue. This is the output of conda list in the venv:

# Name                    Version                   Build  Channel
ca-certificates           2021.7.5             hecd8cb5_1  
certifi                   2021.5.30        py38hecd8cb5_0  
libcxx                    10.0.0                        1  
libffi                    3.3                  hb1e8313_2  
ncurses                   6.2                  h0a44026_1  
openssl                   1.1.1k               h9ed2024_0  
pip                       21.1.3           py38hecd8cb5_0  
python                    3.8.10               h88f2d9e_7  
readline                  8.1                  h9ed2024_0  
setuptools                52.0.0           py38hecd8cb5_0  
sqlite                    3.36.0               hce871da_0  
tk                        8.6.10               hb0a8c7a_0  
wheel                     0.36.2             pyhd3eb1b0_0  
xz                        5.2.5                h1de35cc_0  
zlib                      1.2.11               h1de35cc_3 

Description

When I try to pip install an empty package with the following setup.py, pip seems unable to resolve dependencies and and the installation doesn’t commence.

import setuptools

setuptools.setup(
    install_requires=[
        "tensorflow<=2.5.0",
        "tf-models-official==2.5.0",
    ],
)

However, if I try to install only the dependencies using

pip install "tensorflow<=2.5.0" tf-models-official==2.5.0

it works as expected, which makes me think that this issue is related to setuptools, rather than pip.

Moreover, modifying the setup.py to have either tensorflow==2.5.0, or tf-models-official (without explicit version), also fixes the problem, and in either case pip ends up installing versions 2.5.0 anyway.

Using --no-cache-dir makes no difference.

Expected behavior

To install the package as expected.

How to Reproduce

1. conda create -n tmp python=3.8 -y && conda activate tmp
2. mkdir tmp && cd tmp
3. Create setup.py:

echo "import setuptools
setuptools.setup(
    install_requires=[
        'tensorflow<=2.5.0',
        'tf-models-official==2.5.0',
    ],
)" >> setup.py

1. pip install .

Output

Collecting tensorflow<=2.5.0
  Using cached tensorflow-2.5.0-cp38-cp38-macosx_10_11_x86_64.whl (195.7 MB)
Collecting tf-models-official==2.5.0
  Using cached tf_models_official-2.5.0-py2.py3-none-any.whl (1.6 MB)
Collecting psutil>=5.4.3
  Using cached psutil-5.8.0-cp38-cp38-macosx_10_9_x86_64.whl (236 kB)
Collecting six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting tensorflow-addons
  Using cached tensorflow_addons-0.13.0-cp38-cp38-macosx_10_13_x86_64.whl (511 kB)
Collecting scipy>=0.19.1
  Using cached scipy-1.7.0-cp38-cp38-macosx_10_9_x86_64.whl (31.9 MB)
Collecting pycocotools
  Using cached pycocotools-2.0.2-cp38-cp38-macosx_10_9_x86_64.whl
Collecting py-cpuinfo>=3.3.0
  Using cached py_cpuinfo-8.0.0-py3-none-any.whl
Collecting kaggle>=1.3.9
  Using cached kaggle-1.5.12-py3-none-any.whl
Collecting Pillow
  Using cached Pillow-8.3.1-cp38-cp38-macosx_10_10_x86_64.whl (2.9 MB)
Collecting seqeval
  Using cached seqeval-1.2.2-py3-none-any.whl
Collecting tensorflow-hub>=0.6.0
  Using cached tensorflow_hub-0.12.0-py2.py3-none-any.whl (108 kB)
Collecting pyyaml>=5.1
  Using cached PyYAML-5.4.1-cp38-cp38-macosx_10_9_x86_64.whl (253 kB)
Collecting tensorflow-model-optimization>=0.4.1
  Using cached tensorflow_model_optimization-0.6.0-py2.py3-none-any.whl (211 kB)
Collecting oauth2client
  Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting tf-slim>=1.1.0
  Using cached tf_slim-1.1.0-py2.py3-none-any.whl (352 kB)
Collecting pandas>=0.22.0
  Using cached pandas-1.3.0-cp38-cp38-macosx_10_9_x86_64.whl (11.4 MB)
Collecting opencv-python-headless
  Using cached opencv_python_headless-4.5.3.56-cp38-cp38-macosx_10_15_x86_64.whl (42.6 MB)
Collecting numpy>=1.15.4
  Using cached numpy-1.21.0-cp38-cp38-macosx_10_9_x86_64.whl (16.9 MB)
Collecting tensorflow-datasets
  Using cached tensorflow_datasets-4.3.0-py3-none-any.whl (3.9 MB)
Collecting sacrebleu
  Using cached sacrebleu-1.5.1-py3-none-any.whl (54 kB)
Collecting gin-config
  Using cached gin_config-0.4.0-py2.py3-none-any.whl (46 kB)
Collecting matplotlib
  Using cached matplotlib-3.4.2-cp38-cp38-macosx_10_9_x86_64.whl (7.2 MB)
Collecting sentencepiece
  Using cached sentencepiece-0.1.96-cp38-cp38-macosx_10_6_x86_64.whl (1.1 MB)
Collecting Cython
  Using cached Cython-0.29.24-cp38-cp38-macosx_10_9_x86_64.whl (1.9 MB)
Collecting google-cloud-bigquery>=0.31.0
  Using cached google_cloud_bigquery-2.21.0-py2.py3-none-any.whl (193 kB)
Collecting google-api-python-client>=1.6.7
  Using cached google_api_python_client-2.13.0-py2.py3-none-any.whl (7.1 MB)
Collecting google-auth-httplib2>=0.1.0
  Using cached google_auth_httplib2-0.1.0-py2.py3-none-any.whl (9.3 kB)
Collecting google-auth<2dev,>=1.16.0
  Using cached google_auth-1.33.0-py2.py3-none-any.whl (151 kB)
Collecting google-api-core<2dev,>=1.21.0
  Using cached google_api_core-1.31.0-py2.py3-none-any.whl (93 kB)
Collecting uritemplate<4dev,>=3.0.0
  Using cached uritemplate-3.0.1-py2.py3-none-any.whl (15 kB)
Collecting httplib2<1dev,>=0.15.0
  Using cached httplib2-0.19.1-py3-none-any.whl (95 kB)
Collecting packaging>=14.3
  Using cached packaging-21.0-py3-none-any.whl (40 kB)
Requirement already satisfied: setuptools>=40.3.0 in /Users/laszlokindrat/miniconda3/envs/tmp/lib/python3.8/site-packages (from google-api-core<2dev,>=1.21.0->google-api-python-client>=1.6.7->tf-models-official==2.5.0->UNKNOWN==0.0.0) (52.0.0.post20210125)
Collecting requests<3.0.0dev,>=2.18.0
  Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting protobuf>=3.12.0
  Using cached protobuf-3.17.3-cp38-cp38-macosx_10_9_x86_64.whl (1.0 MB)
Collecting pytz
  Using cached pytz-2021.1-py2.py3-none-any.whl (510 kB)
Collecting googleapis-common-protos<2.0dev,>=1.6.0
  Using cached googleapis_common_protos-1.53.0-py2.py3-none-any.whl (198 kB)
Collecting pyasn1-modules>=0.2.1
  Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting rsa<5,>=3.1.4
  Using cached rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting cachetools<5.0,>=2.0.0
  Using cached cachetools-4.2.2-py3-none-any.whl (11 kB)
Collecting google-cloud-core<2.0dev,>=1.4.1
  Using cached google_cloud_core-1.7.1-py2.py3-none-any.whl (28 kB)
Collecting google-resumable-media<2.0dev,>=0.6.0
  Using cached google_resumable_media-1.3.1-py2.py3-none-any.whl (75 kB)
Collecting proto-plus>=1.10.0
  Using cached proto_plus-1.19.0-py3-none-any.whl (42 kB)
Collecting grpcio<2.0dev,>=1.38.1
  Using cached grpcio-1.38.1-cp38-cp38-macosx_10_10_x86_64.whl (3.9 MB)
Collecting google-crc32c<2.0dev,>=1.0
  Using cached google_crc32c-1.1.2-cp38-cp38-macosx_10_14_x86_64.whl (27 kB)
Collecting cffi>=1.0.0
  Using cached cffi-1.14.6-cp38-cp38-macosx_10_9_x86_64.whl (176 kB)
Collecting pycparser
  Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Collecting pyparsing<3,>=2.4.2
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting urllib3
  Using cached urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
Collecting python-slugify
  Using cached python_slugify-5.0.2-py2.py3-none-any.whl (6.7 kB)
Collecting python-dateutil
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting tqdm
  Using cached tqdm-4.61.2-py2.py3-none-any.whl (76 kB)
Requirement already satisfied: certifi in /Users/laszlokindrat/miniconda3/envs/tmp/lib/python3.8/site-packages (from kaggle>=1.3.9->tf-models-official==2.5.0->UNKNOWN==0.0.0) (2021.5.30)
Collecting pyasn1<0.5.0,>=0.4.6
  Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.2-py3-none-any.whl (59 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.3-py3-none-any.whl (35 kB)
Collecting typing-extensions~=3.7.4
  Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Collecting six
  Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting opt-einsum~=3.3.0
  Using cached opt_einsum-3.3.0-py3-none-any.whl (65 kB)
Collecting numpy>=1.15.4
  Using cached numpy-1.19.5-cp38-cp38-macosx_10_9_x86_64.whl (15.6 MB)
Collecting gast==0.4.0
  Using cached gast-0.4.0-py3-none-any.whl (9.8 kB)
Requirement already satisfied: wheel~=0.35 in /Users/laszlokindrat/miniconda3/envs/tmp/lib/python3.8/site-packages (from tensorflow<=2.5.0->UNKNOWN==0.0.0) (0.36.2)
Collecting wrapt~=1.12.1
  Using cached wrapt-1.12.1-cp38-cp38-macosx_10_9_x86_64.whl
Collecting termcolor~=1.1.0
  Using cached termcolor-1.1.0-py3-none-any.whl
Collecting tensorflow-estimator<2.6.0,>=2.5.0rc0
  Using cached tensorflow_estimator-2.5.0-py2.py3-none-any.whl (462 kB)
INFO: pip is looking at multiple versions of six to determine which version is compatible with other requirements. This could take a while.
Collecting six
  Using cached six-1.14.0-py2.py3-none-any.whl (10 kB)
  Using cached six-1.13.0-py2.py3-none-any.whl (10 kB)
INFO: pip is looking at multiple versions of scipy to determine which version is compatible with other requirements. This could take a while.
Collecting scipy>=0.19.1
  Using cached scipy-1.6.3-cp38-cp38-macosx_10_9_x86_64.whl (30.8 MB)
INFO: pip is looking at multiple versions of six to determine which version is compatible with other requirements. This could take a while.
  Using cached scipy-1.6.2-cp38-cp38-macosx_10_9_x86_64.whl (30.8 MB)
  Using cached scipy-1.6.1-cp38-cp38-macosx_10_9_x86_64.whl (30.8 MB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
  Using cached scipy-1.6.0-cp38-cp38-macosx_10_9_x86_64.whl (30.7 MB)
  Using cached scipy-1.5.4-cp38-cp38-macosx_10_9_x86_64.whl (29.0 MB)
  Using cached scipy-1.5.3-cp38-cp38-macosx_10_9_x86_64.whl (29.0 MB)
  Using cached scipy-1.5.2-cp38-cp38-macosx_10_9_x86_64.whl (28.9 MB)
INFO: pip is looking at multiple versions of scipy to determine which version is compatible with other requirements. This could take a while.
  Using cached scipy-1.5.1-cp38-cp38-macosx_10_9_x86_64.whl (28.9 MB)
  Using cached scipy-1.5.0-cp38-cp38-macosx_10_9_x86_64.whl (28.9 MB)
  Using cached scipy-1.4.1-cp38-cp38-macosx_10_9_x86_64.whl (28.8 MB)
  Using cached scipy-1.4.0-cp38-cp38-macosx_10_9_x86_64.whl (28.8 MB)
  Using cached scipy-1.3.3-cp38-cp38-macosx_10_9_x86_64.whl (28.1 MB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. If you want to abort this run, you can press Ctrl + C to do so. To improve how pip performs, tell us what happened here: https://pip.pypa.io/surveys/backtracking
  Using cached scipy-1.3.2-cp38-cp38-macosx_10_9_x86_64.whl (28.1 MB)
  Using cached scipy-1.3.1.tar.gz (23.6 MB)
  Installing build dependencies ... \

This takes so long that I never observed it finish.

Code of Conduct

• I agree to follow the PSF Code of Conduct

[notatallshaw]

I think can be closed now? I was never able to reproduce issue but https://github.com/pypa/pip/pull/10481 should fix it unless OP states it’s still happening since 21.3.

[laszlo-luminous]

Thanks everyone, looks like upgrading pip to 21.3 solves the issue!