Tests failing to run with BackendUnavailable since pip 19

It seems that since the release of pip 19, Setuptools’ own tests fail to run thus:

setuptools fix_889_and_non-ascii_in_setup.cfg_take_2 $ tox -e py27                                                                                                                       
py27 create: /Users/jaraco/code/main/setuptools/.tox/py27
py27 installdeps: -rtests/requirements.txt
py27 develop-inst: /Users/jaraco/code/main/setuptools
ERROR: invocation failed (exit code 2), logfile: /Users/jaraco/code/main/setuptools/.tox/py27/log/py27-2.log
ERROR: actionid: py27
msg: developpkg
cmdargs: '/Users/jaraco/code/main/setuptools/.tox/py27/bin/pip install --exists-action w -e /Users/jaraco/code/main/setuptools'

DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Obtaining file:///Users/jaraco/code/main/setuptools
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
Exception:
Traceback (most recent call last):
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 176, in main
    status = self.run(options, args)
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 315, in run
    resolver.resolve(requirement_set)
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/resolve.py", line 131, in resolve
    self._resolve_one(requirement_set, req)
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/resolve.py", line 294, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/resolve.py", line 226, in _get_abstract_dist_for
    req, self.require_hashes, self.use_user_site, self.finder,
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/operations/prepare.py", line 382, in prepare_editable_requirement
    abstract_dist.prep_for_dist(finder, self.build_isolation)
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_internal/operations/prepare.py", line 149, in prep_for_dist
    reqs = self.req.pep517_backend.get_requires_for_build_wheel()
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_vendor/pep517/wrappers.py", line 71, in get_requires_for_build_wheel
    'config_settings': config_settings
  File "/Users/jaraco/code/main/setuptools/.tox/py27/lib/python2.7/site-packages/pip/_vendor/pep517/wrappers.py", line 162, in _call_hook
    raise BackendUnavailable
BackendUnavailable

py27 installed: DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.,apipkg==1.5,atomicwrites==1.2.1,attrs==18.2.0,configparser==3.7.1,contextlib2==0.5.5,coverage==4.5.2,enum34==1.1.6,execnet==1.5.0,flake8==3.6.0,funcsigs==1.0.2,futures==3.2.0,importlib-metadata==0.8,mccabe==0.6.1,mock==2.0.0,more-itertools==5.0.0,path.py==11.5.0,pathlib2==2.3.3,pbr==5.1.1,pluggy==0.8.1,py==1.7.0,pycodestyle==2.4.0,pyflakes==2.0.0,pytest==3.10.1,pytest-cov==2.6.1,pytest-fixture-config==1.4.0,pytest-flake8==1.0.3,pytest-shutil==1.4.0,pytest-virtualenv==1.4.0,scandir==1.9.0,six==1.12.0,termcolor==1.1.0,virtualenv==16.3.0,zipp==0.3.3
________________________________________________________________________________________ summary ________________________________________________________________________________________
ERROR:   py27: InvocationError for command /Users/jaraco/code/main/setuptools/.tox/py27/bin/pip install --exists-action w -e /Users/jaraco/code/main/setuptools (see /Users/jaraco/code/main/setuptools/.tox/py27/log/py27-2.log) (exited with code 2)

I suspect this is related to #1642 if not just another manifestation of the same issue.

[pganssle]

Thinking about this, even if pip --no-binary :all: switches to resolving self-referential source distributions with wheels, if they don’t handle long-range cycles, we’re still going to break --no-binary :all: if we depend on wheel as part of the PEP 517 build.

That basically means that we cannot have any dependencies in build-system.requires, and we’ll need to vendor wheel, so I think we have three options:

1. Specify 'setuptools' and 'wheel' in our build-system.requires and let pip fix --no-binary :all: or not, as desired.
2. Opt out of PEP 517 forever, meaning that we will never be installable by a PEP 517-only frontend except as a wheel.
3. Vendor wheel in our source tree and wait for PEP 517 to be extended to allow for self-bootstrapping builds.

I don’t think number 3 is going to happen, so we’re really choosing between options 1 and 2. I’m in favor of option 1, but we may want to give pip some fair warning that we’ll be doing it, since it will likely cause at least some breakages.

[benoit-pierre]

No, it’s because the versions of pip being used are different…