Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

pex does not support musllinux wheels

See original GitHub issue

PEP 656 introduced the musllinux tag to support Alpine Linux and other cases where CPython does not use glibc.

I have confirmed that using python3 and pip in a venv a musllinux wheel is downloaded.

pip install 'psycopg2-binary'
Collecting psycopg2-binary
  Downloading psycopg2_binary-2.9.3-cp310-cp310-musllinux_1_1_x86_64.whl (1.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.9/1.9 MB 8.5 MB/s eta 0:00:00
Installing collected packages: psycopg2-binary
Successfully installed psycopg2-binary-2.9.3
WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available.
You should consider upgrading via the '/root/venv/bin/python3 -m pip install --upgrade pip' command.

However using pex results in:

pex --pip-version 22.2.2 --resolver-version pip-2020-resolver --no-build 'psycopg2-binary'
Failed to find compatible interpreter on path /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin.

Examined the following interpreters:
1.) /usr/bin/python3.10 CPython==3.10.5

No interpreter compatible with the requested constraints was found:

  A distribution for psycopg2-binary could not be resolved for /usr/bin/python3.10.
  Found 1 distribution for psycopg2-binary that do not apply:
  1.) The wheel tags for psycopg2-binary 2.9.3 are cp310-cp310-musllinux_1_1_x86_64 which do not match the supported tags of /usr/bin/python3.10:
  cp310-cp310-linux_x86_64
  ... 34 more ...

To reproduce the above run the following to get alpine linux with python3 in docker.

$ docker run --rm -it alpine:3.16
# apk update
# apk add python3

I suspect that this is related to how the vendored packaging library is 20.9 but the project added support for musllinux wheels in 21.0. Unfortunately that release dropped support for Python 2 as well.

A possible fix would be to have --pip-version 22.2.2 also imply a newer version of the packaging library.

Issue Analytics

State:
Created a year ago
Comments:6 (5 by maintainers)

Top GitHub Comments

1reaction

jsiroiscommented, Oct 5, 2022

$ tox -epackage
package create: /home/jsirois/dev/pantsbuild/jsirois-pex/.tox/package
package installdeps: flit, flit_core<3.7.0, pygments, toml
package installed: certifi==2022.9.24,charset-normalizer==2.1.1,docutils==0.19,flit==3.6.0,flit_core==3.6.0,idna==3.4,Pygments==2.13.0,requests==2.28.1,toml==0.10.2,tomli==2.0.1,tomli_w==1.0.0,urllib3==1.26.12
package run-test-pre: PYTHONHASHSEED='1665990705'
package run-test: commands[0] | python scripts/package.py
Building Pex PEX to `dist/pex` ...
Built Pex PEX @ v2.1.108-1-g1a1b9aa:
sha256: a6dd1f19f8f4048834cb6aece46674262a761147d1eea63e25489cf5e75c1f77
  size: 4065209
____________________________________________________________________________________________________ summary _____________________________________________________________________________________________________
  package: commands succeeded
  congratulations :)
$ docker run --rm -it -v $PWD/dist:/dist alpine:3.16
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
/ # apk update
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/community/x86_64/APKINDEX.tar.gz
v3.16.2-221-ge7097e0782 [https://dl-cdn.alpinelinux.org/alpine/v3.16/main]
v3.16.2-232-ga327e65f26 [https://dl-cdn.alpinelinux.org/alpine/v3.16/community]
OK: 17033 distinct packages available
/ # apk add python3
(1/13) Installing libbz2 (1.0.8-r1)
(2/13) Installing expat (2.4.9-r0)
(3/13) Installing libffi (3.4.2-r1)
(4/13) Installing gdbm (1.23-r0)
(5/13) Installing xz-libs (5.2.5-r1)
(6/13) Installing libgcc (11.2.1_git20220219-r2)
(7/13) Installing libstdc++ (11.2.1_git20220219-r2)
(8/13) Installing mpdecimal (2.5.1-r1)
(9/13) Installing ncurses-terminfo-base (6.3_p20220521-r0)
(10/13) Installing ncurses-libs (6.3_p20220521-r0)
(11/13) Installing readline (8.1.2-r0)
(12/13) Installing sqlite-libs (3.38.5-r0)
(13/13) Installing python3 (3.10.5-r0)
Executing busybox-1.35.0-r17.trigger
OK: 57 MiB in 27 packages
/ # python3 /dist/pex --pip-version 22.2.2 --resolver-version pip-2020-resolver --no-build 'psycopg2-binary'
Python 3.10.5 (main, Jul 25 2022, 15:52:08) [GCC 11.2.1 20220219] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import psycopg2
>>> psycopg2.__file__
'/root/.pex/installed_wheels/c3ae8e75eb7160851e59adc77b3a19a976e50622e44fd4fd47b8b18208189d42/psycopg2_binary-2.9.3-cp310-cp310-musllinux_1_1_x86_64.whl/psycopg2/__init__.py'
>>>
now exiting InteractiveConsole...
/ #
$ python -mpex -oempty.pex
$ zipinfo -1 empty.pex | grep _vendored/packaging
.bootstrap/pex/vendor/_vendored/packaging_20_9/
.bootstrap/pex/vendor/_vendored/packaging_20_9/__init__.py
.bootstrap/pex/vendor/_vendored/packaging_20_9/packaging/
.bootstrap/pex/vendor/_vendored/packaging_20_9/packaging/__about__.py
.bootstrap/pex/vendor/_vendored/packaging_20_9/packaging/__init__.py
.bootstrap/pex/vendor/_vendored/packaging_20_9/packaging/_compat.py
.bootstrap/pex/vendor/_vendored/packaging_20_9/packaging/_structures.py
.bootstrap/pex/vendor/_vendored/packaging_20_9/packaging/_typing.py
.bootstrap/pex/vendor/_vendored/packaging_20_9/packaging/markers.py
.bootstrap/pex/vendor/_vendored/packaging_20_9/packaging/py.typed
.bootstrap/pex/vendor/_vendored/packaging_20_9/packaging/requirements.py
.bootstrap/pex/vendor/_vendored/packaging_20_9/packaging/specifiers.py
.bootstrap/pex/vendor/_vendored/packaging_20_9/packaging/tags.py
.bootstrap/pex/vendor/_vendored/packaging_20_9/packaging/utils.py
.bootstrap/pex/vendor/_vendored/packaging_20_9/packaging/version.py
.bootstrap/pex/vendor/_vendored/packaging_20_9/pyparsing.py
.bootstrap/pex/vendor/_vendored/packaging_21_3/
.bootstrap/pex/vendor/_vendored/packaging_21_3/__init__.py
.bootstrap/pex/vendor/_vendored/packaging_21_3/packaging/
.bootstrap/pex/vendor/_vendored/packaging_21_3/packaging/__about__.py
.bootstrap/pex/vendor/_vendored/packaging_21_3/packaging/__init__.py
.bootstrap/pex/vendor/_vendored/packaging_21_3/packaging/_manylinux.py
.bootstrap/pex/vendor/_vendored/packaging_21_3/packaging/_musllinux.py
.bootstrap/pex/vendor/_vendored/packaging_21_3/packaging/_structures.py
.bootstrap/pex/vendor/_vendored/packaging_21_3/packaging/markers.py
.bootstrap/pex/vendor/_vendored/packaging_21_3/packaging/py.typed
.bootstrap/pex/vendor/_vendored/packaging_21_3/packaging/requirements.py
.bootstrap/pex/vendor/_vendored/packaging_21_3/packaging/specifiers.py
.bootstrap/pex/vendor/_vendored/packaging_21_3/packaging/tags.py
.bootstrap/pex/vendor/_vendored/packaging_21_3/packaging/utils.py
.bootstrap/pex/vendor/_vendored/packaging_21_3/packaging/version.py
.bootstrap/pex/vendor/_vendored/packaging_21_3/pyparsing.py

1reaction

jsiroiscommented, Oct 5, 2022

Ok, this is a good bit easier than I expected.