Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Move micropip to a separate repo in the org

See original GitHub issue

Continuing with the effort of making the code base more approachable (https://github.com/pyodide/pyodide/issues/2014), I would propose to move micropip to a separate repo in the Pyodide org.

Advantages

mich faster iteration cycle to change anything in micropip. CI should take minutes not an hour as currently.
for new contributors, it’s also much easier to start since it’s a pure Python package they don’t need to deal with the rest of the build system.
can update micropip for a given Pyodide release by loading the pure Python wheel from PyPi with pyodide.loadPackage. The release schedule of micropip can become independent from Pyodide.
there are several other projects in this space (e.g. emscripten-forge, https://github.com/ethanhs/python-wasm) and this would make it easier to reuse and contribute back fixes if necessary. There is still the FFI question which is more difficult, but right now even if emscripten-forge wanted to support installing Pyodide packages (and assuming binary compatibility) they have no other way short of vendoring it. Similarly, I’m hoping this could encourage some of the micropip related fixes from JupyterLite to be contributed upstream.

Advantages

a bit more setup work, but we have done this for a lot of packages now, so it’s mostly copy-pasting
makes it harder to make large changes across different components of Pyodide, though I think micropip is pretty stable now and there are rarely PRs that need to change both the internals and micropip at the same time.

Issue Analytics

State:
Created a year ago
Reactions:1
Comments:5 (5 by maintainers)

Top GitHub Comments

1reaction

hoodmanecommented, Sep 13, 2022

I approve of this proposal. The idea of updating micropip more often than Pyodide is particularly exciting.

One thought is that we should consider having a way to override or patch _compat_in_pyodide in case something has to change in that file. micropip uses the following four internals:

API.loadBinaryFile
API.loadDynlib
API.repodata_packages
API.repodata_info

We should mark these to note that they are used by micropip. We might want to add some specific functions to pyodide to get repodata_packages and repodata_info. Also, loadDynlib takes a boolean as the second parameter, maybe we should change it to an options object for forwards compatibility.

0reactions

rthcommented, Sep 18, 2022

I created https://github.com/pyodide/micropip and this will be resolved with https://github.com/pyodide/pyodide/pull/3122