`ModuleNotFoundError: No module named 'distutils.msvccompiler'` in package build CI

🐛 Bug

Several different packages have failed with ModuleNotFoundError: No module named 'distutils.msvccompiler'. See for instance: https://app.circleci.com/pipelines/github/hoodmane/pyodide/3667/workflows/dd548c6b-f1c8-40d3-88f5-e71573bf4934/jobs/43716

@ryanking13 any ideas? Maybe we need to pin something?

[hoodmane]

Is Pyodide really not able to use two different versions of a pure Python package at build time?

We can do that yes. We use pypabuild with build isolation. I think we may need to add some extra logic that inserts a version upper bound if it’s consistent. Something like in pseudo code:

setuptools_json = fetch_json("pypi.org/setuptools/json")
setuptools_vers = get_versions(setuptools_json)

lt_65_vers = [v for v in setuptools_vers if v in setuptools_constrait and v.major<65]
if less_than_65_vers:
   setuptools_constraint = "setuptools=="+max(less_than_65_vers)

[ryanking13]

I think we need to pin the setuptools version. Setuptools embeds distutils in its source code and it seems like they removed some of msvc modules recently:

https://github.com/pypa/setuptools/pull/3505