New `--no-build-isolation` check in Pip 22.1 broke `oldest-supported-numpy`

Description

The change in gh-10886 to check for presence of build dependencies when building with --no-build-isolation broke users of oldest-supported-numpy (see https://github.com/scipy/oldest-supported-numpy/issues/53).

For context, oldest-supported-numpy is a meta-package which ensures that the correct numpy version is installed at build time, handling platform and Python-version specific details so those don’t have to be replicated in each package which depends on NumPy’s C API. Also important: it is a PyPI-specific package; other packaging systems have different ways of dealing with NumPy ABI issues and do not necessarily use the same NumPy versions to build against as the NumPy team recommends as the default for wheels.

A use case like this does not seem to have been considered at all in gh-10886 or the issue for it (gh-9794). It doesn’t make sense to have a metapackage like this installed in many cases, nor is it desired to force users to build with exactly one numpy version - in CI for example, it makes perfect sense to test with multiple versions; the only requirement is that the version at runtime is >= the version used at build time.

I don’t see a good way to fix this up in either oldest-supported-numpy or in the pyproject.toml files from users of that package, and I think that therefore the change in Pip’s behavior should be reverted.

Expected behavior

I expect the reproducer to result in a successful build of scikit-learn. Users must be able to specify oldest-supported-numpy in their build dependencies without the new check being triggered.

This is probably not the only use case that broke, but in case you really want to keep this check: an alternative could be to special-case oldest-supported-numpy inside the code performing the check; instead just check that a version of numpy is installed.

pip version

22.1

Python version

all

OS

all

How to Reproduce

Example for one of a number of packages that are affected:

pip install cython scipy 'setuptools<60'
git clone https://github.com/scikit-learn/scikit-learn
cd scikit-learn
pip install --no-build-isolation .

Output

Processing /Users/rgommers/code/tmp/scikit-learn
ERROR: Some build dependencies for file:///Users/rgommers/code/tmp/scikit-learn are missing: 'oldest-supported-numpy'.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[uranusjr]

I think we should not validate by default, and add a flag to enable validation.

[jakirkham]

Yeah for validation in some of these cases, we are not terribly concerned with validation at build time, but do care at install time. Typically we are handling this by running things like pip check once things are in place (so after pip install and possibly other steps like bundling artifacts).