Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Problems building venvs from certain distributions.

See original GitHub issue

@asherf reports two cases:

Exhibited by selenium 4.1.2

$ python3.8 -mpex selenium==4.1.2 --venv
Traceback (most recent call last):
  File "/home/jsirois/.pyenv/versions/3.8.12/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/jsirois/.pyenv/versions/3.8.12/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/tmpyim2gyjr/__main__.py", line 86, in <module>
    bootstrap_pex(__entry_point__)
  File "/tmp/tmpyim2gyjr/.bootstrap/pex/pex_bootstrapper.py", line 513, in bootstrap_pex
    venv_pex = ensure_venv(pex.PEX(entry_point, interpreter=target))
  File "/tmp/tmpyim2gyjr/.bootstrap/pex/pex_bootstrapper.py", line 461, in ensure_venv
    shebang = populate_venv(
  File "/tmp/tmpyim2gyjr/.bootstrap/pex/venv/pex.py", line 120, in populate_venv
    record_provenance(_populate_deps(venv, pex, venv_python, symlink))
  File "/tmp/tmpyim2gyjr/.bootstrap/pex/venv/pex.py", line 116, in record_provenance
    for src, dst in src_to_dst:
  File "/tmp/tmpyim2gyjr/.bootstrap/pex/venv/pex.py", line 216, in _populate_deps
    for src, dst in record.reinstall(venv, symlink=symlink, rel_extra_path=rel_extra_path):
  File "/tmp/tmpyim2gyjr/.bootstrap/pex/pep_376.py", line 414, in reinstall
    raise ReinstallError(
pex.pep_376.ReinstallError: Cannot re-install file from /home/jsirois/.pex/installed_wheels/591b3b9ffc0450f97d2e0ef3559dd39b3259b707/selenium-4.1.2-py3-none-any.whl/selenium-4.1.2.dist-info/RECORD:1, refusing to install to absolute path /selenium/__init__.py.

Exhibited by greenlet 1.1.2

$ python3.8 -mpex greenlet==1.1.2 --venv
Traceback (most recent call last):
  File "/home/jsirois/.pyenv/versions/3.8.12/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/jsirois/.pyenv/versions/3.8.12/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/tmp/tmpb6vlpu58/__main__.py", line 86, in <module>
    bootstrap_pex(__entry_point__)
  File "/tmp/tmpb6vlpu58/.bootstrap/pex/pex_bootstrapper.py", line 513, in bootstrap_pex
    venv_pex = ensure_venv(pex.PEX(entry_point, interpreter=target))
  File "/tmp/tmpb6vlpu58/.bootstrap/pex/pex_bootstrapper.py", line 461, in ensure_venv
    shebang = populate_venv(
  File "/tmp/tmpb6vlpu58/.bootstrap/pex/venv/pex.py", line 120, in populate_venv
    record_provenance(_populate_deps(venv, pex, venv_python, symlink))
  File "/tmp/tmpb6vlpu58/.bootstrap/pex/venv/pex.py", line 116, in record_provenance
    for src, dst in src_to_dst:
  File "/tmp/tmpb6vlpu58/.bootstrap/pex/venv/pex.py", line 216, in _populate_deps
    for src, dst in record.reinstall(venv, symlink=symlink, rel_extra_path=rel_extra_path):
  File "/tmp/tmpb6vlpu58/.bootstrap/pex/pep_376.py", line 423, in reinstall
    raise ReinstallError(
pex.pep_376.ReinstallError: Cannot re-install file from /home/jsirois/.pex/installed_wheels/673770872757fe48f96719d7818b2bffb6afba16/greenlet-1.1.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl/greenlet-1.1.2.dist-info/RECORD:1, path does not match --target scheme: ../../../../home/jsirois/dev/pantsbuild/jsirois-pex/include/site/python3.8/greenlet/greenlet.h

Issue Analytics

State:
Created 2 years ago
Comments:6 (6 by maintainers)

Top GitHub Comments

1reaction

jsiroiscommented, Mar 10, 2022

Alright, it turns out 1 was untrue, the pypy layout is not broken for the jupyter-server share/ data files case - it was just me wrongly assuming the nbconvert transitive dep carried those, when it was in fact ipykernel. I’ve managed to get all this working, but it’s hefty in LOC. Simple idea, lots of layout details to get right.

1reaction

jsiroiscommented, Mar 9, 2022

Alright, I’ve got this working save for a handful of ITs and a handful of unit tests using --prefix. Bootstrapping vendoring with this new scheme was a bear, but that is sussed. I’ll be putting in enough time tomorrow to get a PR and a Pex release out for this (https://github.com/jsirois/pex/commit/10b4400723b122fac337762b1b58552905b3ac7d).