Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
pex 2.1.101 issue: virtualenv is not valid
See original GitHub issueLooks like there’s an issue in pex 2.1.101.
How to reproduce: anywhere you’ve got docker
docker run -it --rm centos:7 /bin/bash -c "yum -y install python3 && python3.6 -m pip install pex==2.1.101 && pex --python python3.6 pyyaml==5.4.1"
Output:
Loaded plugins: fastestmirror, ovl
Determining fastest mirrors
* base: download.cf.centos.org
* extras: download.cf.centos.org
* updates: download.cf.centos.org
base | 3.6 kB 00:00:00
extras | 2.9 kB 00:00:00
updates | 2.9 kB 00:00:00
(1/4): base/7/x86_64/group_gz | 153 kB 00:00:00
(2/4): extras/7/x86_64/primary_db | 247 kB 00:00:00
(3/4): base/7/x86_64/primary_db | 6.1 MB 00:00:00
(4/4): updates/7/x86_64/primary_db | 16 MB 00:00:01
Resolving Dependencies
--> Running transaction check
---> Package python3.x86_64 0:3.6.8-18.el7 will be installed
--> Processing Dependency: python3-libs(x86-64) = 3.6.8-18.el7 for package: python3-3.6.8-18.el7.x86_64
--> Processing Dependency: python3-setuptools for package: python3-3.6.8-18.el7.x86_64
--> Processing Dependency: python3-pip for package: python3-3.6.8-18.el7.x86_64
--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-18.el7.x86_64
--> Running transaction check
---> Package python3-libs.x86_64 0:3.6.8-18.el7 will be installed
--> Processing Dependency: libtirpc.so.1()(64bit) for package: python3-libs-3.6.8-18.el7.x86_64
---> Package python3-pip.noarch 0:9.0.3-8.el7 will be installed
---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed
--> Running transaction check
---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=================================================================================================================================================================================================================
Package Arch Version Repository Size
=================================================================================================================================================================================================================
Installing:
python3 x86_64 3.6.8-18.el7 updates 70 k
Installing for dependencies:
libtirpc x86_64 0.2.4-0.16.el7 base 89 k
python3-libs x86_64 3.6.8-18.el7 updates 6.9 M
python3-pip noarch 9.0.3-8.el7 base 1.6 M
python3-setuptools noarch 39.2.0-10.el7 base 629 k
Transaction Summary
=================================================================================================================================================================================================================
Install 1 Package (+4 Dependent packages)
Total download size: 9.3 M
Installed size: 48 M
Downloading packages:
warning: /var/cache/yum/x86_64/7/updates/packages/python3-3.6.8-18.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY ] 0.0 B/s | 0 B --:--:-- ETA
Public key for python3-3.6.8-18.el7.x86_64.rpm is not installed
(1/5): python3-3.6.8-18.el7.x86_64.rpm | 70 kB 00:00:00
Public key for libtirpc-0.2.4-0.16.el7.x86_64.rpm is not installed
(2/5): libtirpc-0.2.4-0.16.el7.x86_64.rpm | 89 kB 00:00:00
(3/5): python3-setuptools-39.2.0-10.el7.noarch.rpm | 629 kB 00:00:00
(4/5): python3-pip-9.0.3-8.el7.noarch.rpm | 1.6 MB 00:00:00
(5/5): python3-libs-3.6.8-18.el7.x86_64.rpm | 6.9 MB 00:00:01
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 6.3 MB/s | 9.3 MB 00:00:01
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Importing GPG key 0xF4A80EB5:
Userid : "CentOS-7 Key (CentOS 7 Official Signing Key) <security@centos.org>"
Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5
Package : centos-release-7-9.2009.0.el7.centos.x86_64 (@CentOS)
From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : libtirpc-0.2.4-0.16.el7.x86_64 1/5
Installing : python3-setuptools-39.2.0-10.el7.noarch 2/5
Installing : python3-pip-9.0.3-8.el7.noarch 3/5
Installing : python3-3.6.8-18.el7.x86_64 4/5
Installing : python3-libs-3.6.8-18.el7.x86_64 5/5
Verifying : libtirpc-0.2.4-0.16.el7.x86_64 1/5
Verifying : python3-setuptools-39.2.0-10.el7.noarch 2/5
Verifying : python3-libs-3.6.8-18.el7.x86_64 3/5
Verifying : python3-3.6.8-18.el7.x86_64 4/5
Verifying : python3-pip-9.0.3-8.el7.noarch 5/5
Installed:
python3.x86_64 0:3.6.8-18.el7
Dependency Installed:
libtirpc.x86_64 0:0.2.4-0.16.el7 python3-libs.x86_64 0:3.6.8-18.el7 python3-pip.noarch 0:9.0.3-8.el7 python3-setuptools.noarch 0:39.2.0-10.el7
Complete!
WARNING: Running pip install with root privileges is generally not a good idea. Try `__main__.py install --user` instead.
Collecting pex==2.1.101
Downloading https://files.pythonhosted.org/packages/77/11/696975448606b1b29ebc877aae43035537cb1ddc2e6a8f85a2ee448ce2e2/pex-2.1.101-py2.py3-none-any.whl (2.7MB)
100% |################################| 2.7MB 352kB/s
Installing collected packages: pex
Successfully installed pex-2.1.101
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/pex/pep_376.py", line 423, in from_prefix_install
site_packages = find_site_packages_dir(venv_dir=prefix_dir, interpreter=interpreter)
File "/usr/local/lib/python3.6/site-packages/pex/venv/virtualenv.py", line 105, in find_site_packages_dir
venv_dir=venv_dir, site_packages_dir=site_packages_dir
pex.venv.virtualenv.InvalidVirtualenvError: The virtualenv at /root/.pex/installed_wheels/4e0583d24c881e14342eaf4ec5fbc97f934b999a6828693a99157fde912540cc/PyYAML-5.4.1-cp36-cp36m-manylinux1_x86_64.whl.85f410fb476a45dfaeb71ee64fdd4120 is not valid. The expected site-packages directory at /root/.pex/installed_wheels/4e0583d24c881e14342eaf4ec5fbc97f934b999a6828693a99157fde912540cc/PyYAML-5.4.1-cp36-cp36m-manylinux1_x86_64.whl.85f410fb476a45dfaeb71ee64fdd4120/lib/python3.6/site-packages does not exist.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/pex", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python3.6/site-packages/pex/bin/pex.py", line 782, in main
env=env,
File "/usr/local/lib/python3.6/site-packages/pex/result.py", line 103, in catch
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/pex/bin/pex.py", line 802, in do_main
options=options,
File "/usr/local/lib/python3.6/site-packages/pex/bin/pex.py", line 701, in build_pex
preserve_log=resolver_configuration.preserve_log,
File "/usr/local/lib/python3.6/site-packages/pex/resolver.py", line 949, in resolve
ignore_errors=ignore_errors, max_parallel_jobs=max_parallel_jobs
File "/usr/local/lib/python3.6/site-packages/pex/resolver.py", line 718, in install_distributions
max_jobs=max_parallel_jobs,
File "/usr/local/lib/python3.6/site-packages/pex/jobs.py", line 541, in execute_parallel
yield spawn_result.spawned_job.await_result()
File "/usr/local/lib/python3.6/site-packages/pex/jobs.py", line 220, in await_result
job.wait()
File "/usr/local/lib/python3.6/site-packages/pex/jobs.py", line 81, in wait
self._finalize_job()
File "/usr/local/lib/python3.6/site-packages/pex/jobs.py", line 139, in _finalize_job
self._finalizer(self._process.returncode)
File "/usr/local/lib/python3.6/site-packages/pex/pip/tool.py", line 715, in fixup_install
interpreter=interpreter,
File "/usr/local/lib/python3.6/site-packages/pex/pep_376.py", line 427, in from_prefix_install
"{prefix_dir}: {err}".format(prefix_dir=prefix_dir, err=e)
pex.pep_376.RecordNotFoundError: Could not find a site-packages directory under installation prefix /root/.pex/installed_wheels/4e0583d24c881e14342eaf4ec5fbc97f934b999a6828693a99157fde912540cc/PyYAML-5.4.1-cp36-cp36m-manylinux1_x86_64.whl.85f410fb476a45dfaeb71ee64fdd4120: The virtualenv at /root/.pex/installed_wheels/4e0583d24c881e14342eaf4ec5fbc97f934b999a6828693a99157fde912540cc/PyYAML-5.4.1-cp36-cp36m-manylinux1_x86_64.whl.85f410fb476a45dfaeb71ee64fdd4120 is not valid. The expected site-packages directory at /root/.pex/installed_wheels/4e0583d24c881e14342eaf4ec5fbc97f934b999a6828693a99157fde912540cc/PyYAML-5.4.1-cp36-cp36m-manylinux1_x86_64.whl.85f410fb476a45dfaeb71ee64fdd4120/lib/python3.6/site-packages does not exist.
It looks like pyyaml is getting installed, but under lib64 instead of lib where pex is looking based on the output:
[root@67f5c04250bf /]# ls /root/.pex/installed_wheels/4e0583d24c881e14342eaf4ec5fbc97f934b999a6828693a99157fde912540cc/PyYAML-5.4.1-cp36-cp36m-manylinux1_x86_64.whl.ef84b8851b22467dbbcc39c67ed88c0d/
lib64
Issue Analytics
- State:
- Created a year ago
- Comments:8 (6 by maintainers)
Top Results From Across the Web
Building .pex files — pex 2.1.101 documentation
This command creates a pex file containing pex and requests, using the console script named “pex”, saving it in ~/bin/pex. At this point,...
Read more >pex
pex is a library for generating .pex (Python EXecutable) files which are executable Python environments in the spirit of virtualenvs. pex is an...
Read more >pex Changelog - pyup.io
This release brings two fixes for ``--venv`` mode PEXes. * Only insert "" to head of sys.path if a venv PEX runs in...
Read more >'virtualenv' is not recognized as an internal or external ...
steps: - go to where you want create django app on that folder. then run this command on command prompt : python -m...
Read more >Python Virtual Environments: A Primer
In this tutorial, you'll learn how to work with Python's venv module to create and manage separate virtual environments for your Python ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Alright, I think the best path forward is to revert back to the prior naive approach, but instead of simply walking the prefix directory for the 1st ‘site-packages’ subdir, just collect them all. The current code handles then checking each found
site-packagesfor the install and so this should both continue to solve #1861 and robustly find whatever-the-schemesite-packes(assuming its calledsite-packages!).Hrm, thanks @sfc-gh-pbennes. I’ve got some learning to do here.