pip 21.3+ in-tree-builds with setuptools, do not work with read-only source trees

Description

We are using pip as part of our build pipeline within a project containing a mix of languages. In order to keep everything clean and reproducible, we mount the source code read-only in a docker container and run the build steps there.

Even it’s a mix of languages and tools, this approach used to work fine until now, when pip as the only of all the tools involved suddenly tried to touch the source tree during build.

I have read though the discussion in https://github.com/pypa/pip/issues/7555 that led to this change, but I feel like the point that there might be good reasons not to touch the source directory didn’t get enough attention:

• You want to keep source code and build artifacts separate
• You don’t want to clutter your scm
• Files created within the docker container are usually owned by root. So in our setup, our source directory would be cluttered with files owned by root after the container exits
• You want to make sure that no build script accidentally modifies the source code at build time. This is a very important point especially in large projects where it’s hard to find side-effects in build scripts

Expected behavior

All other major packaging tools I know offer an option to chose a build directory that is separate from the source dir. I would expect that pip works in a similar way.

pip version

21.3

Python version

3.10

OS

ubuntu

How to Reproduce

Create a python package, mount read-only into a docker container and try to let pip install it, e.g:

mkdir readonly
echo "from setuptools import setup\nsetup(name='hello', version='1.0.0')" > readonly/setup.py
docker run -ti --rm -v `pwd`/readonly:/src:ro python:3.10 /bin/bash -c "pip install --upgrade pip; pip install /src"
rm -rf readonly

Output

Building wheels for collected packages: hello
  Building wheel for hello (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/local/bin/python -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/src/setup.py'"'"'; __file__='"'"'/src/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-gx0vxrjn
       cwd: /src/
  Complete output (8 lines):
  running bdist_wheel
  running build
  installing to build/bdist.linux-x86_64/wheel
  running install
  running install_egg_info
  running egg_info
  creating hello.egg-info
  error: could not create 'hello.egg-info': Read-only file system
  ----------------------------------------
  ERROR: Failed building wheel for hello
  Running setup.py clean for hello
Failed to build hello

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[psigen]

https://pip.pypa.io/en/stable/topics/local-project-installs/#build-artifacts

Changed in version 22.1: The --use-deprecated=out-of-tree-build option has been removed.

Welp, the use-deprecated option is now deprecated. 😱

Is there any remaining workaround for the increasingly common use-case of wanting to pip install off a read-only mounted directory (like a network mounted FS or docker volume)? Currently I’ve just pinned the older pip and crossed my fingers that a solution is forthcoming 🤞.

[psigen]

@dhirschfeld I’m not sure how to escalate this further.

I’m pretty sure that this is affecting many people using docker volume mounts in their CI pipelines, since the sane default is to use a read-only source volume from a build reproducibility and caching standpoint. I’m just hoping someone can help.