noexec /tmp causes pip install to miss chmod +x on scripts included in wheel file

Environment

pip version: pip 19.0.3 Python version: python 3.6 OS: Oracle Linux 7.6

tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noexec,relatime)

Description Pip won’t install a script as executable if /tmp is mounted as noexec. I have a wheel file that was created with setuptools. Pip unpacks it into /tmp and chmod the file to 755 which is correct and executable. However, when pip goes to copy the file from the tmp staging dir to the destination it uses access(file, X_OK) to check if the file is executable rather than looking at the mode flags. the access() returns “EACCES (permission denied)” and pip skips the chmod step.

Broken: chmod("/tmp/pip-install-lzeaps2y/YYY/YYY-0.3.4.data/scripts/export", 0755) = 0 ... stat("/tmp/pip-install-lzeaps2y/YYY/YYY-0.3.4.data/scripts/export", {st_mode=S_IFREG|0755, st_size=3593, ...}) = 0 access("/tmp/pip-install-lzeaps2y/YYY/YYY-0.3.4.data/scripts/export", X_OK) = -1 EACCES (Permission denied) stat("/home/XXX/pt/bin/export", {st_mode=S_IFREG|0644, st_size=3593, ...}) = 0

Working: chmod("/tmp/pip-install-bfuur148/YYY/YYY-0.3.4.data/scripts/export", 0775) = 0 ... stat("/tmp/pip-install-bfuur148/YYY/YYY-0.3.4.data/scripts/export", {st_mode=S_IFREG|0775, st_size=3593, ...}) = 0 access("/tmp/pip-install-bfuur148/YYY/YYY-0.3.4.data/scripts/export", X_OK) = 0 stat("/tmp/pip-install-bfuur148/YYY/YYY-0.3.4.data/scripts/export", {st_mode=S_IFREG|0775, st_size=3593, ...}) = 0 chmod("/home/XXX/pt/bin/export", 0100775) = 0 stat("/home/XXX/pt/bin/export", {st_mode=S_IFREG|0775, st_size=3593, ...}) = 0

Expected behavior

How to Reproduce Enable tmpfs /tmp with noexec mount -t tmpfs -o rw,nosuid,nodev,noexec,relatime tmpfs /mnt

Create a python wheel file with a script that just echos test

cat << EOF > testbin
#!/bin/bash
echo test
EOF

cat << EOF > setup.py
import setuptools
setuptools.setup(
    name="example-pkg",
    version="0.0.1",
    scripts=["testbin"]
)
EOF

virtualenv --python python36 pt
source pt/bin/activate

python setup.py bdist_wheel
pip install dist/example_pkg-0.0.1-py3-none-any.whl

ls -l pt/bin

Output

Broken output shows the testbin is not +x

(pt) [XXX@YYY]$ cat << EOF > testbin
> #!/bin/bash
> echo test
> EOF
(pt) [XXX@YYY]$
(pt) [XXX@YYY]$ cat << EOF > setup.py
> import setuptools
> setuptools.setup(
>     name="example-pkg",
>     version="0.0.1",
>     scripts=["testbin"]
> )
> EOF
(pt) [XXX@YYY]$
(pt) [XXX@YYY]$ virtualenv --python python36 pt
Running virtualenv with interpreter /usr/bin/python36
Using base prefix '/usr'
New python executable in /backups/tmp/pt/bin/python36
Also creating executable in /backups/tmp/pt/bin/python
Installing setuptools, pip, wheel...done.
(pt) [XXX@YYY]$ source pt/bin/activate
(pt) [XXX@YYY]$
(pt) [XXX@YYY]$ python setup.py bdist_wheel
running bdist_wheel
running build
running build_scripts
copying testbin -> build/scripts-3.6
changing mode of build/scripts-3.6/testbin from 644 to 755
installing to build/bdist.linux-x86_64/wheel
running install
running install_egg_info
running egg_info
writing example_pkg.egg-info/PKG-INFO
writing dependency_links to example_pkg.egg-info/dependency_links.txt
writing top-level names to example_pkg.egg-info/top_level.txt
reading manifest file 'example_pkg.egg-info/SOURCES.txt'
writing manifest file 'example_pkg.egg-info/SOURCES.txt'
Copying example_pkg.egg-info to build/bdist.linux-x86_64/wheel/example_pkg-0.0.1-py3.6.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/example_pkg-0.0.1.data
creating build/bdist.linux-x86_64/wheel/example_pkg-0.0.1.data/scripts
copying build/scripts-3.6/testbin -> build/bdist.linux-x86_64/wheel/example_pkg-0.0.1.data/scripts
changing mode of build/bdist.linux-x86_64/wheel/example_pkg-0.0.1.data/scripts/testbin to 755
creating build/bdist.linux-x86_64/wheel/example_pkg-0.0.1.dist-info/WHEEL
(pt) [XXX@YYY]$ pip install dist/example_pkg-0.0.1-py3-none-any.whl
Processing ./dist/example_pkg-0.0.1-py3-none-any.whl
Installing collected packages: example-pkg
Successfully installed example-pkg-0.0.1
(pt) [XXX@YYY]$ ls -l pt/bin/testbin
-rw-r--r-- 1 XXX access-login 22 Mar 28 19:51 pt/bin/testbin

[bsolomon1124]

OK, I am more than happy to keep it simple 😉 @chrahunt.

[chrahunt]

We don’t need to feel like we have to do what os.access does. Setting all x bits when any is set seems reasonable to me.

We could replace this condition with something like:

if st.st_mode & (stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH):
    ...

This has nice properties:

1. likely to be the intended outcome - we’re just trying to make sure included scripts are executable if they are set as such
2. easy to reason about
3. minimal code
4. easy to test