AssertionError no _distutils when running get-pip with Setuptools and USE_DISTUTILS=local

Somehow related: Since setuptools v60 at least on Debian running get-pip.py over an existing pip + setuptools fails, when the Debian python3-distutils package is installed, e.g. as dependency of the Python development headers package:

/usr/local/lib/python3.9/dist-packages/_distutils_hack/__init__.py:24: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
  warnings.warn(
/usr/local/lib/python3.9/dist-packages/_distutils_hack/__init__.py:36: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
Traceback (most recent call last):
  File "/tmp/DietPi-Software/./get-pip.py", line 27081, in <module>
    main()
  File "/tmp/DietPi-Software/./get-pip.py", line 139, in main
    bootstrap(tmpdir=tmpdir)
  File "/tmp/DietPi-Software/./get-pip.py", line 120, in bootstrap
    args = determine_pip_install_arguments()
  File "/tmp/DietPi-Software/./get-pip.py", line 65, in determine_pip_install_arguments
    import setuptools  # noqa
  File "/usr/local/lib/python3.9/dist-packages/setuptools/__init__.py", line 8, in <module>
    import _distutils_hack.override  # noqa: F401
  File "/usr/local/lib/python3.9/dist-packages/_distutils_hack/override.py", line 1, in <module>
    __import__('_distutils_hack').do_override()
  File "/usr/local/lib/python3.9/dist-packages/_distutils_hack/__init__.py", line 73, in do_override
    ensure_local_distutils()
  File "/usr/local/lib/python3.9/dist-packages/_distutils_hack/__init__.py", line 61, in ensure_local_distutils
    assert '_distutils' in core.__file__, core.__file__
AssertionError: /usr/lib/python3.9/distutils/core.py

This _distutils_hack comes from get-pip.py. Looks like removing any use and detangling distutils from pip in favour of setuptools would solve both issues.

_Originally posted by @MichaIng in https://github.com/pypa/pip/issues/10742#issuecomment-1003741034_

[jaraco]

Setuptools 60 intentionally introduced its local copy of distutils as default. As an escape hatch, you should be able to set the environment variable SETUPTOOLS_USE_DISTUTILS=stdlib while running get-pip to bypass that behavior.

I know the distutils hack has an exclusion for when running under pip. That was, however, to account for pip’s use of distutils. In this case, get-pip is specifically importing setuptools. Importing setuptools causes the more aggressive form of requiring distutils, where if the local copy of distutils isn’t found, it fails with the reported error.

So the question is - what is it about this environment that triggers this behavior? In particular, how is it that this code doesn’t ensure that any existing distutils is unloaded, then loaded from Setuptools?

Can someone create a set of docker steps that will replicate the issue?

[grv87]

@jaraco, have you finished your work on this issue? Please, take a look at pypa/setuptools#3439