`vendoring` is broken, due to a cyclic dependency during license fetching

Well, the lack of maintainance of the license fetching logic in vendoring has come to bite us. 😃

flit recently established a cyclic dependency, by depending on tomli: see https://github.com/takluyver/flit/issues/451 and https://flit.readthedocs.io/en/latest/bootstrap.html.

We get licenses from sdists in vendoring (which means building metadata / wheels – leading to https://github.com/pradyunsg/vendoring/issues/1). Since flit is no longer bootstrappable through regular mechanisms, it’d be best to switch to using wheels for the license fetch phase.

This is the tracking issue for actually fixing this, and adopting the fix in our workflow.

[takluyver]

backend-path is working correctly for flit_core, as far as I’m concerned. It’s set here, and flit_core can be built from source with no problem:

pip wheel --no-binary :all: --no-deps flit_core

The issue arises because flit_core has a runtime dependency (not a build dependency) on tomli. So tomli’s build dependency on flit_core means it has a transitive build dependency on itself. Building flit_core is fine, using it for tomli is what falls down.

Another possible workaround would be to have a special build backend in tomli sdists - e.g. bundling flit_core in the sdists - and using backend-path in tomli’s pyproject.toml to use that. That would work, as far as I know, but I don’t think it should be on tomli to do special workarounds for bootstrapping - if we need special cases, they should be in flit_core.

Yet another possibility is that pip could detect such a cycle - we’re trying to build tomli, and we’ve found that we need tomli - and use a wheel in an isolated build env to break it, unless you pass some extra option --no-wheels-not-even-to-break-build-cycles. But that’s probably a lot of work for a hopefully very specific situation.

[takluyver]

I don’t particularly mind doing that - but I’d guess that there’s a large overlap between the people who can’t use a pre-built wheel and the people who reject bundled packages, so I don’t know if bundling would help many people.