How to bootstrap setuptools from source

Hi, I’m a developer for the Spack package manager. We’ve had a setuptools package that has worked fine for a while now, and the vast majority of our Python packages depend on it. However, I went to update to the latest version of setuptools and noticed that it now depends on six, packaging, and appdirs. Unfortunately all 3 of these packages also depend on setuptools. six and packaging can fall back on distutils.core if necessary, but the setup.py for appdirs has a hard setuptools import.

It seems to me like it is no longer possible to build setuptools from source. Is this correct??

[jaraco]

With Setuptools 36, it once again vendors its dependencies.

[dstufft]

So there exist systems where given a set of dependencies (python and otherwise) build everything from source is a requirement. This gets complicated when you have circular dependencies. Imagine a situation where you want to install setuptools:

1. You have a source tarball of setuptools, so you build a setuptools binary (it can be a wheel or some other system it doesn’t matter). This works with the PR in #982 just fine.
2. You discover that setuptools requires packaging, six, appdirs.
3. You attempt to build appdirs from source, which requires setuptools to build, so you attempt to go and install setuptools, which fails because while you have a setuptools binary package you don’t have it’s runtime packages.

AFAICT having setuptools depend on anything which itself depends on setuptools (build or runtime) is always going to make an assumption that you have somehow managed to have a copy of setuptools + all dependencies already built already available. Any system which cannot have that assumption will not work.

This isn’t unique to setuptools either. Even with PEP 518 a build tool is not going to be able to depend on itself directly or transitively without breaking from scratch source rebuilds.