Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
pip install nondeterministic order even for explicitly provided wheels
See original GitHub issueDescription
Subsequent calls to pip install do not execute in the same order, even when wheels are vendored, and no index is used.
This seems unnecessarily nondeterministic (to me), and makes it harder than necessary to reproduce bugs (including bugs in pip).
Expected behavior
No response
pip version
22.0.2
Python version
Ptyhon 3.10
OS
any (presumably); in practice: ubuntu
How to Reproduce
(areyoudeterministic) me:/tmp/areyoudeterministic$ cat requirements.txt
Django==4.0.*
pytz==2022.1
whitenoise==6.0.0
sqlparse==0.4.2
cfenv==0.5.3
Authlib==1.0.1
requests==2.27.1
djangorestframework==3.13.1
djangorestframework-datatables==0.7.0
cryptography
pandas==1.5.0
reportlab==3.6.11
gunicorn==20.1.0
mysqlclient==2.1.0
django-migration-linter
cfenv==0.5.3
django-auth-ldap==4.0.0
(areyoudeterministic) me:/tmp/areyoudeterministic$ pip wheel --no-binary=:none: -r requirements.txt -w vendor
Collecting Django==4.0.*
Using cached Django-4.0.8-py3-none-any.whl (8.0 MB)
[..]
Saved ./vendor/setuptools-65.5.0-py3-none-any.whl
(areyoudeterministic) me:/tmp/areyoudeterministic$ python -m venv .
(areyoudeterministic) me:/tmp/areyoudeterministic$ pip install -r requirements.txt --find-links=vendor --no-index
Looking in links: vendor
Processing ./vendor/Django-4.0.8-py3-none-any.whl
Processing ./vendor/pytz-2022.1-py2.py3-none-any.whl
Processing ./vendor/whitenoise-6.0.0-py3-none-any.whl
Processing ./vendor/sqlparse-0.4.2-py3-none-any.whl
Processing ./vendor/cfenv-0.5.3-py2.py3-none-any.whl
Processing ./vendor/Authlib-1.0.1-py2.py3-none-any.whl
Processing ./vendor/requests-2.27.1-py2.py3-none-any.whl
Processing ./vendor/djangorestframework-3.13.1-py3-none-any.whl
Processing ./vendor/djangorestframework_datatables-0.7.0-py2.py3-none-any.whl
Processing ./vendor/cryptography-38.0.3-cp36-abi3-manylinux_2_28_x86_64.whl
Processing ./vendor/pandas-1.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Processing ./vendor/reportlab-3.6.11-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Processing ./vendor/gunicorn-20.1.0-py3-none-any.whl
Processing ./vendor/mysqlclient-2.1.0-cp310-cp310-linux_x86_64.whl
Processing ./vendor/django_migration_linter-4.1.0-py3-none-any.whl
Processing ./vendor/django_auth_ldap-4.0.0-py3-none-any.whl
Processing ./vendor/asgiref-3.5.2-py3-none-any.whl
Processing ./vendor/furl-2.1.3-py2.py3-none-any.whl
Processing ./vendor/idna-3.4-py3-none-any.whl
Processing ./vendor/urllib3-1.26.12-py2.py3-none-any.whl
Processing ./vendor/certifi-2022.9.24-py3-none-any.whl
Processing ./vendor/charset_normalizer-2.0.12-py3-none-any.whl
Processing ./vendor/python_dateutil-2.8.2-py2.py3-none-any.whl
Processing ./vendor/numpy-1.23.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Processing ./vendor/Pillow-9.3.0-cp310-cp310-manylinux_2_28_x86_64.whl
Requirement already satisfied: setuptools>=3.0 in ./lib/python3.10/site-packages (from gunicorn==20.1.0->-r requirements.txt (line 13)) (59.6.0)
Processing ./vendor/python_ldap-3.4.3-cp310-cp310-linux_x86_64.whl
Processing ./vendor/cffi-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Processing ./vendor/toml-0.10.2-py2.py3-none-any.whl
Processing ./vendor/appdirs-1.4.4-py2.py3-none-any.whl
Processing ./vendor/pycparser-2.21-py2.py3-none-any.whl
Processing ./vendor/six-1.16.0-py2.py3-none-any.whl
Processing ./vendor/orderedmultidict-1.0.1-py2.py3-none-any.whl
Processing ./vendor/pyasn1_modules-0.2.8-py2.py3-none-any.whl
Processing ./vendor/pyasn1-0.4.8-py2.py3-none-any.whl
Installing collected packages: pytz, pyasn1, appdirs, whitenoise, urllib3, toml, sqlparse, six, pycparser, pyasn1-modules, pillow, numpy, mysqlclient, idna, gunicorn, charset-normalizer, certifi, asgiref, requests, reportlab, python-ldap, python-dateutil, orderedmultidict, Django, cffi, pandas, furl, djangorestframework, django-migration-linter, django-auth-ldap, cryptography, djangorestframework-datatables, cfenv, Authlib
Successfully installed Authlib-1.0.1 Django-4.0.8 appdirs-1.4.4 asgiref-3.5.2 certifi-2022.9.24 cfenv-0.5.3 cffi-1.15.1 charset-normalizer-2.0.12 cryptography-38.0.3 django-auth-ldap-4.0.0 django-migration-linter-4.1.0 djangorestframework-3.13.1 djangorestframework-datatables-0.7.0 furl-2.1.3 gunicorn-20.1.0 idna-3.4 mysqlclient-2.1.0 numpy-1.23.4 orderedmultidict-1.0.1 pandas-1.5.0 pillow-9.3.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 python-dateutil-2.8.2 python-ldap-3.4.3 pytz-2022.1 reportlab-3.6.11 requests-2.27.1 six-1.16.0 sqlparse-0.4.2 toml-0.10.2 urllib3-1.26.12 whitenoise-6.0.0
(areyoudeterministic) me:/tmp/areyoudeterministic$ rm bin/ include/ lib lib64/ -rf
(areyoudeterministic) me:/tmp/areyoudeterministic$ python -m venv .
(areyoudeterministic) me:/tmp/areyoudeterministic$ pip install -r requirements.txt --find-links=vendor --no-index
Looking in links: vendor
Processing ./vendor/Django-4.0.8-py3-none-any.whl
Processing ./vendor/pytz-2022.1-py2.py3-none-any.whl
Processing ./vendor/whitenoise-6.0.0-py3-none-any.whl
Processing ./vendor/sqlparse-0.4.2-py3-none-any.whl
Processing ./vendor/cfenv-0.5.3-py2.py3-none-any.whl
Processing ./vendor/Authlib-1.0.1-py2.py3-none-any.whl
Processing ./vendor/requests-2.27.1-py2.py3-none-any.whl
Processing ./vendor/djangorestframework-3.13.1-py3-none-any.whl
Processing ./vendor/djangorestframework_datatables-0.7.0-py2.py3-none-any.whl
Processing ./vendor/cryptography-38.0.3-cp36-abi3-manylinux_2_28_x86_64.whl
Processing ./vendor/pandas-1.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Processing ./vendor/reportlab-3.6.11-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Processing ./vendor/gunicorn-20.1.0-py3-none-any.whl
Processing ./vendor/mysqlclient-2.1.0-cp310-cp310-linux_x86_64.whl
Processing ./vendor/django_migration_linter-4.1.0-py3-none-any.whl
Processing ./vendor/django_auth_ldap-4.0.0-py3-none-any.whl
Processing ./vendor/asgiref-3.5.2-py3-none-any.whl
Processing ./vendor/furl-2.1.3-py2.py3-none-any.whl
Processing ./vendor/charset_normalizer-2.0.12-py3-none-any.whl
Processing ./vendor/idna-3.4-py3-none-any.whl
Processing ./vendor/urllib3-1.26.12-py2.py3-none-any.whl
Processing ./vendor/certifi-2022.9.24-py3-none-any.whl
Processing ./vendor/numpy-1.23.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Processing ./vendor/python_dateutil-2.8.2-py2.py3-none-any.whl
Processing ./vendor/Pillow-9.3.0-cp310-cp310-manylinux_2_28_x86_64.whl
Requirement already satisfied: setuptools>=3.0 in ./lib/python3.10/site-packages (from gunicorn==20.1.0->-r requirements.txt (line 13)) (59.6.0)
Processing ./vendor/python_ldap-3.4.3-cp310-cp310-linux_x86_64.whl
Processing ./vendor/cffi-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Processing ./vendor/appdirs-1.4.4-py2.py3-none-any.whl
Processing ./vendor/toml-0.10.2-py2.py3-none-any.whl
Processing ./vendor/pycparser-2.21-py2.py3-none-any.whl
Processing ./vendor/six-1.16.0-py2.py3-none-any.whl
Processing ./vendor/orderedmultidict-1.0.1-py2.py3-none-any.whl
Processing ./vendor/pyasn1-0.4.8-py2.py3-none-any.whl
Processing ./vendor/pyasn1_modules-0.2.8-py2.py3-none-any.whl
Installing collected packages: pytz, pyasn1, appdirs, whitenoise, urllib3, toml, sqlparse, six, pycparser, pyasn1-modules, pillow, numpy, mysqlclient, idna, gunicorn, charset-normalizer, certifi, asgiref, requests, reportlab, python-ldap, python-dateutil, orderedmultidict, Django, cffi, pandas, furl, djangorestframework, django-migration-linter, django-auth-ldap, cryptography, djangorestframework-datatables, cfenv, Authlib
Successfully installed Authlib-1.0.1 Django-4.0.8 appdirs-1.4.4 asgiref-3.5.2 certifi-2022.9.24 cfenv-0.5.3 cffi-1.15.1 charset-normalizer-2.0.12 cryptography-38.0.3 django-auth-ldap-4.0.0 django-migration-linter-4.1.0 djangorestframework-3.13.1 djangorestframework-datatables-0.7.0 furl-2.1.3 gunicorn-20.1.0 idna-3.4 mysqlclient-2.1.0 numpy-1.23.4 orderedmultidict-1.0.1 pandas-1.5.0 pillow-9.3.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 python-dateutil-2.8.2 python-ldap-3.4.3 pytz-2022.1 reportlab-3.6.11 requests-2.27.1 six-1.16.0 sqlparse-0.4.2 toml-0.10.2 urllib3-1.26.12 whitenoise-6.0.0
(areyoudeterministic) me:/tmp/areyoudeterministic$ pip --version
pip 22.0.2 from /tmp/areyoudeterministic/lib/python3.10/site-packages/pip (python 3.10)
Note the positioning of furl-2.1.3-py2.py3-none-any.whl and idna-3.4-py3-none-any.whl in the 2 subsequent runs.
Output
No response
Code of Conduct
- I agree to follow the PSF Code of Conduct.
Issue Analytics
- State:
- Created a year ago
- Comments:14 (14 by maintainers)
Top Results From Across the Web
Wheel caching and non-deterministic builds - Packaging
So, we have a non-deterministic build process. However, pip always caches the built wheel. The user later attempts to load the missing library ......
Read more >wheel Documentation - Read the Docs
The above pip install command will replace the current interpreter's installed wheel package with the development package being tested. If you ...
Read more >pip install — pip 10.0.0.dev0 documentation
Identify the base requirements. The user supplied arguments are processed here. Resolve dependencies. What will be installed is determined here. Build wheels.
Read more >What is the meaning of "Failed building wheel for X" in pip ...
In your case, you're missing the wheel package so pip is unable to build wheels from source dists. if you want to explicitly...
Read more >pip install — pip 8.1.1 documentation
Since version 7.0 pip supports controlling the command line options given to setup.py via requirements files. This disables the use of wheels ( ......
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
If curiosity and spare time come together I might just do that
Instead of getting back into an extensive discussion about the nature of determistic vs non-deterministic order in data structures… I reckon it’d be a better investment of effort to investigate the specific cause of this issue.