vendoring logic subtly breaks later usage of unvendored system versions after importing pkg_resources.extern

In theory, if pkg_resources.extern is devendored, it should defer to the system installation of e.g. packaging, which means that:

>>> packaging.version
<module 'pip._vendor.packaging.version' from '/usr/lib/python3.6/site-packages/packaging/version.py'>
>>> pkg_resources.extern.packaging.version
<module 'pip._vendor.packaging.version' from '/usr/lib/python3.6/site-packages/packaging/version.py'>

It would be expected that I could then compare the results, since they’re the same thing.

>>> pkg_resources.extern.packaging.version.parse('3.0') == packaging.version.parse('2.0')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: '>' not supported between instances of 'Version' and 'Version'

Oops?

For comparison, this works fine when using the similarly devendored pip.

>>> pip._vendor.packaging.version
<module 'packaging.version' from '/usr/lib/python3.6/site-packages/packaging/version.py'>
>>> packaging.version
<module 'packaging.version' from '/usr/lib/python3.6/site-packages/packaging/version.py'>
>>> pip._vendor.packaging.version.parse('3.0') > packaging.version.parse('2.0')
True

Note that pkg_resources must not be devendored from pip, or else it must be deleted from vendor/__init__.py, since otherwise it will be automatically imported. Even importing pkg_resources causes pip to fail as well.

>>> import pip._vendor.packaging.version
>>> import packaging.version
>>> pip._vendor.packaging.version.parse('3.0') > packaging.version.parse('2.0')
True

vs.

>>> import pip._vendor.packaging.version
>>> import pkg_resources.extern
>>> import packaging.version
>>> pip._vendor.packaging.version.parse('3.0') > packaging.version.parse('2.0')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: '>' not supported between instances of 'Version' and 'Version'

https://github.com/pypa/pip/issues/5429#issuecomment-391070003 suggested the problem is deleting the global module unconditionally, whenever something vendored is in use. This strikes me as incredibly dangerous.

[mgorny]

If anyone else’s still looking for a “good enough” hack, I’ve ended up taking unbundling one step further and removing setuptools.extern and pkg_resources.extern entirely:

	rm -r */extern || die
	find -name '*.py' -exec sed \
		-e 's:from \w*[.]extern ::' -e 's:\w*[.]extern[.]::' \
		-i {} + || die

[mgorny]

Ironically, this bug is now causing 60.6.0 tests to fail:

_______________________________________________________ TestDepends.testRequire _______________________________________________________
[gw10] linux -- Python 3.9.10 /tmp/portage/dev-python/setuptools-60.6.0/work/setuptools-60.6.0-pypy3/install/usr/bin/pypy3

self = <setuptools.tests.test_setuptools.TestDepends object at 0x000055ee2b945d70>

    @needs_bytecode
    def testRequire(self):
        req = Require('Json', '1.0.3', 'json')
    
        assert req.name == 'Json'
        assert req.module == 'json'
>       assert req.requested_version == version.Version('1.0.3')
E       AssertionError: assert <Version('1.0.3')> == <Version('1.0.3')>
E         +<Version('1.0.3')>
E         -<Version('1.0.3')>

req        = <setuptools.depends.Require object at 0x000055ee2ea540c8>
self       = <setuptools.tests.test_setuptools.TestDepends object at 0x000055ee2b945d70>

/tmp/portage/dev-python/setuptools-60.6.0/work/setuptools-60.6.0/setuptools/tests/test_setuptools.py:94: AssertionError