build_meta:__legacy__ config_settings not compatible with pip options

Summary

Running pip wheel --global-option ARG calls setup.py ARG bdist_wheel, but running setuptools.build_meta.build_wheel(config_settings={"--global-option": ["ARG"]}) calls setup.py bdist_wheel ARG! Can we make pip and setuptools behave the same?

Details

setuptools.build_meta:__legacy__ (I will drop __legacy__ since it is just a wrapper around build_meta) supports the --global-option value in config_settings, but places it at a different location in constructed setup.py invocations than pip does. pip supports a --build-option parameter for post-command args, which matches the current setuptools.build_meta --global-option behavior.

This can be shown with the following script

t.sh

#!/bin/sh

cd "$(mktemp -d)"

python -V

echo ============== Installing prereqs ==============
python -m venv env
./env/bin/python -m pip install --upgrade pip setuptools wheel pep517
./env/bin/python -m pip freeze --all

cat <<EOF > setup.py
import sys
from setuptools import setup
print(sys.argv)
setup(name="example")
EOF

echo ============== Running pip wheel ==============
./env/bin/python -m pip wheel \
    --disable-pip-version-check \
    --global-option --quiet \
    --build-option --plat-name=foo \
    -v .

cat <<EOF > pyproject.toml
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta:__legacy__"
EOF

# invoke backend directly because pip doesn't currently support
# `config_settings` (see pypa/pip#5771)
cat <<EOF > build.py
from pep517.envbuild import build_wheel

config_settings = {
    "--global-option": ["--quiet"],
    "--build-option": ["--plat-name=foo"],
}

build_wheel(".", ".", config_settings)
EOF

echo ============== Running setuptools.build_meta ==============
./env/bin/python build.py

Here is the output of the script on my machine (Ubuntu 18.04, Python 3.8.0)

Output

Python 3.8.0
============== Installing prereqs ==============
Collecting pip
  Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl
Collecting setuptools
  Using cached https://files.pythonhosted.org/packages/54/28/c45d8b54c1339f9644b87663945e54a8503cfef59cf0f65b3ff5dd17cf64/setuptools-42.0.2-py2.py3-none-any.whl
Collecting wheel
  Using cached https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl
Collecting pep517
  Using cached https://files.pythonhosted.org/packages/f4/9b/82910c0f01f29c7bdd8fc4306ed03e1742256612e2cfca8f05ebb21958ab/pep517-0.8.1-py2.py3-none-any.whl
Collecting toml (from pep517)
  Using cached https://files.pythonhosted.org/packages/a2/12/ced7105d2de62fa7c8fb5fce92cc4ce66b57c95fb875e9318dba7f8c5db0/toml-0.10.0-py2.py3-none-any.whl
Installing collected packages: pip, setuptools, wheel, toml, pep517
  Found existing installation: pip 19.2.3
    Uninstalling pip-19.2.3:
      Successfully uninstalled pip-19.2.3
  Found existing installation: setuptools 41.2.0
    Uninstalling setuptools-41.2.0:
      Successfully uninstalled setuptools-41.2.0
Successfully installed pep517-0.8.1 pip-19.3.1 setuptools-42.0.2 toml-0.10.0 wheel-0.33.6
pep517==0.8.1
pip==19.3.1
setuptools==42.0.2
toml==0.10.0
wheel==0.33.6
============== Running pip wheel ==============
/tmp/user/1000/tmp.wXBXX6X4IO/env/lib/python3.8/site-packages/pip/_internal/commands/wheel.py:115: UserWarning: Disabling all use of wheels due to the use of --build-options / --global-options / --install-options.
  cmdoptions.check_install_build_global(options)
Created temporary directory: /tmp/user/1000/pip-ephem-wheel-cache-prtj7mer
Created temporary directory: /tmp/user/1000/pip-req-tracker-9vt687l5
Created requirements tracker '/tmp/user/1000/pip-req-tracker-9vt687l5'
Created temporary directory: /tmp/user/1000/pip-wheel-2cug6fql
Processing /tmp/user/1000/tmp.wXBXX6X4IO
  Created temporary directory: /tmp/user/1000/pip-req-build-ukgu7746
  Added file:///tmp/user/1000/tmp.wXBXX6X4IO to build tracker '/tmp/user/1000/pip-req-tracker-9vt687l5'
    Running setup.py (path:/tmp/user/1000/pip-req-build-ukgu7746/setup.py) egg_info for package from file:///tmp/user/1000/tmp.wXBXX6X4IO
    Running command python setup.py egg_info
    ['/tmp/user/1000/pip-req-build-ukgu7746/setup.py', 'egg_info', '--egg-base', '/tmp/user/1000/pip-req-build-ukgu7746/pip-egg-info']
    running egg_info
    creating /tmp/user/1000/pip-req-build-ukgu7746/pip-egg-info/example.egg-info
    writing /tmp/user/1000/pip-req-build-ukgu7746/pip-egg-info/example.egg-info/PKG-INFO
    writing dependency_links to /tmp/user/1000/pip-req-build-ukgu7746/pip-egg-info/example.egg-info/dependency_links.txt
    writing top-level names to /tmp/user/1000/pip-req-build-ukgu7746/pip-egg-info/example.egg-info/top_level.txt
    writing manifest file '/tmp/user/1000/pip-req-build-ukgu7746/pip-egg-info/example.egg-info/SOURCES.txt'
    reading manifest file '/tmp/user/1000/pip-req-build-ukgu7746/pip-egg-info/example.egg-info/SOURCES.txt'
    writing manifest file '/tmp/user/1000/pip-req-build-ukgu7746/pip-egg-info/example.egg-info/SOURCES.txt'
  Source in /tmp/user/1000/pip-req-build-ukgu7746 has version 0.0.0, which satisfies requirement example==0.0.0 from file:///tmp/user/1000/tmp.wXBXX6X4IO
  Removed example==0.0.0 from file:///tmp/user/1000/tmp.wXBXX6X4IO from build tracker '/tmp/user/1000/pip-req-tracker-9vt687l5'
Building wheels for collected packages: example
  Created temporary directory: /tmp/user/1000/pip-wheel-zvwa52k1
  Building wheel for example (setup.py) ...   Destination directory: /tmp/user/1000/pip-wheel-zvwa52k1
  Running command /tmp/user/1000/tmp.wXBXX6X4IO/env/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/user/1000/pip-req-build-ukgu7746/setup.py'"'"'; __file__='"'"'/tmp/user/1000/pip-req-build-ukgu7746/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' --quiet bdist_wheel -d /tmp/user/1000/pip-wheel-zvwa52k1 --plat-name=foo
  ['/tmp/user/1000/pip-req-build-ukgu7746/setup.py', '--quiet', 'bdist_wheel', '-d', '/tmp/user/1000/pip-wheel-zvwa52k1', '--plat-name=foo']
done
  Created wheel for example: filename=example-0.0.0-py3-none-foo.whl size=991 sha256=2ba2632248ed833ce1744288e2b271aed52d917c9e5e513c1bca6ab86ae9b6c3
  Stored in directory: /tmp/user/1000/tmp.wXBXX6X4IO
Successfully built example
Cleaning up...
  Removing source in /tmp/user/1000/pip-req-build-ukgu7746
Removed build tracker '/tmp/user/1000/pip-req-tracker-9vt687l5'
============== Running setuptools.build_meta ==============
['/tmp/user/1000/tmp.wXBXX6X4IO/env/lib/python3.8/site-packages/pep517/_in_process.py', 'egg_info', '--quiet']
running egg_info
creating example.egg-info
writing example.egg-info/PKG-INFO
writing dependency_links to example.egg-info/dependency_links.txt
writing top-level names to example.egg-info/top_level.txt
writing manifest file 'example.egg-info/SOURCES.txt'
reading manifest file 'example.egg-info/SOURCES.txt'
writing manifest file 'example.egg-info/SOURCES.txt'
['/tmp/user/1000/tmp.wXBXX6X4IO/env/lib/python3.8/site-packages/pep517/_in_process.py', 'bdist_wheel', '--dist-dir', '/tmp/user/1000/tmp.wXBXX6X4IO/tmpdhxoqdx2', '--quiet']
running bdist_wheel
running build
installing to build/bdist.linux-x86_64/wheel
running install
running install_egg_info
running egg_info
writing example.egg-info/PKG-INFO
writing dependency_links to example.egg-info/dependency_links.txt
writing top-level names to example.egg-info/top_level.txt
reading manifest file 'example.egg-info/SOURCES.txt'
writing manifest file 'example.egg-info/SOURCES.txt'
Copying example.egg-info to build/bdist.linux-x86_64/wheel/example-0.0.0-py3.8.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/example-0.0.0.dist-info/WHEEL
creating '/tmp/user/1000/tmp.wXBXX6X4IO/tmpdhxoqdx2/example-0.0.0-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'example-0.0.0.dist-info/METADATA'
adding 'example-0.0.0.dist-info/WHEEL'
adding 'example-0.0.0.dist-info/top_level.txt'
adding 'example-0.0.0.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel

Note that with the pip invocations, the args are:

['/tmp/user/1000/pip-req-build-ukgu7746/setup.py', 'egg_info', '--egg-base', '/tmp/user/1000/pip-req-build-ukgu7746/pip-egg-info']
['/tmp/user/1000/pip-req-build-ukgu7746/setup.py', '--quiet', 'bdist_wheel', '-d', '/tmp/user/1000/pip-wheel-zvwa52k1', '--plat-name=foo']

whereas invoking setuptools.build_meta results in

['/tmp/user/1000/tmp.wXBXX6X4IO/env/lib/python3.8/site-packages/pep517/_in_process.py', 'egg_info', '--quiet']
['/tmp/user/1000/tmp.wXBXX6X4IO/env/lib/python3.8/site-packages/pep517/_in_process.py', 'bdist_wheel', '--dist-dir', '/tmp/user/1000/tmp.wXBXX6X4IO/tmpdhxoqdx2', '--quiet']

The differences:

1. passing --global-options to setup.py egg_info
   1. pip doesn’t do this - it is tracked by pypa/pip#4383
   2. setuptools.build_meta passes them after egg_info
2. passing --global-options to setup.py bdist_wheel
   1. pip places them before bdist_wheel
   2. setuptools.build_meta places them after bdist_wheel
3. passing --build-options to setup.py bdist_wheel
   1. pip places them after bdist_wheel
   2. setuptools.build_meta does not use this value

For where in code this is happening, see:

• setuptools.build_meta here, where config_settings["--global-option"] comes after the command name
• pip here and here, where we create the initial set of arguments followed by the name of the command followed by build_options

We would like to eventually transition away from having setuptools-specific code in pip by e.g. delegating to the legacy backend (pypa/pip#5204). In that context, can we discuss how to resolve these differences and pass the arguments to the backend appropriately?

[abravalheri]

Maybe this is just a UI question about pip, but usually when you call pip install on a package, it installs a bunch of stuff, so presumably it would pass these arbitrary arguments to all of them.

@pganssle, @jaraco, this happens true if pip is installing several packages from sdist or directly from their source trees.

If all the packages you are installing are available as wheels, config_settings will not take effect.

If you using pip to install the package under development (which I suppose represents the majority of the use cases for this feature), chances are that all the dependencies will come from pre-built wheels and only your local package will be affected by config_settings. The same considerations should be valid for pip wheel.

I think the best thing we can do now is to go with @jaraco’s proposal in https://github.com/pypa/setuptools/commit/b519c18fad92cc6d3d2f31012362246a8467cd41, for 2 reasons:

1. In my opinion, the existing implementation does not make sense under any perspective:

   • If the motivation for including it was to mirror pip’s options, the existing code it is a bug.
   • If we are concerned about the same option being applied to multiple packages at once, it should not be there
   • If --global-options refers to --verbose, --quiet, --no-user-cfg etc…), then it makes more sense to add them to argv before the command name. Otherwise, naming the flag --global-options makes little sense.

2. config_settings is a scape hatch stablished by PEP 517 to be using during “build time”, not “installation time”. Therefore it makes more sense to think about build more then pip (after all setuptools own docs recommend using python -m build as a replacement for python setup.py build_wheel). In this scenario, building one package at the time is the norm.

[pganssle]

I am very uneasy about the whole --global-option situation, and the arbitrary options passed to setuptools.build_meta via the config argument.

Maybe this is just a UI question about pip, but usually when you call pip install on a package, it installs a bunch of stuff, so presumably it would pass these arbitrary arguments to all of them.

I guess I can’t think of a better “escape hatch” in general, but I think we really need to step up our game with extras, to make it easier to avoid complicated installation options ever being passed to pip.