List of "projects using plum"?

It would be useful to have a list of projects that use plum, or even other Python MD libraries. Best would be OSS so we can see exactly how it is used.

(Maybe there is a way to do a pypi reverse-dependency search?)

Some things that such a list might help with:

• My main motivation: Say I want to introduce MD in a garden-variety existing large project. The response of the other devs will be “no”. It would be useful to point to code bases using MD that didn’t cause the end of the world, or even helped.
• Even if I think plum is the best MD library for a particular project, but the best large example uses, say multipledispatch, it’s worth knowing. If I can convince others (and most importantly myself!) that this is was a good choice, it is a relatively small jump to say plum is better in this case. The harder question is whether MD is useful at all.
• What are the design problems and choices peculiar to Python? For example, integration with Python’s OO model.

[wesselb]

I also know of Geometric Kernels, which uses Plum to provide a backend-agnostic implementation (something that works with PyTorch/TF/JAX/etc).

There is fastdispatch, which I believe is eventually to be used in fastcore.

There is PySSAGES, which I’m not familiar with, but which seems to use Plum. (@InnocentBug)

There is EMLP, which I’m also not familiar with, but which seems to use Plum to some extent.

Perhaps there are a few more projects using it.

I also have a few projects of my own which are quite intensively using Plum:

• LAB: provide types to make things backend-agnostic, so it works with PyTorch/TF/JAX/etc.
• Matrix: provide structured matrix types to accelerate linear algebra, building on LAB
• MLKernels: positive-definite kernels, buidling on LAB and Matrix
• Stheno: probabilistic programming with Gaussian processes, building on LAB, Matrix, and MLKernels
• Varz: backend-agnostic hyperparameter optimisation, buidling on LAB
• GPCM, OILMM, GPAR: implementations of some machine learning models, using all of the above
• neuralprocesses: a compositional framework for constructing neural processes, also using a fair few above packages

Once we decide on a way forward with the documentation (see #56, #57), let’s put together this list!

[jlapeyre]

NetKet, a library for Jax/Flax Machine Learning targeted at quantum Physics

Great. This is exactly the kind of thing I am looking for.

Requoting what you quoted:

The response of the other devs will be “no”.

Turns out that’s not universal. In fact, I brought it up with qiskit devs. Actually several are rather positively disposed towards it. But, naturally they don’t want to jump in blindly. And they did point out some concerns, partly in my example code (lack of clarity, performance, etc.)

this talk The Unreasonable effectiveness of Multiple Dispatch

There are a few versions of this talk (a good version is by both Stefan and Jeff at a compiler (I think) conference). I really like them. But these are talks by the people who are heavily invested in this approach. So another dev, especially one not familiar with the concept, would be right to be cautious about accepting the arguments. Seeing actually Python projects that are benefiting is more convincing, I think.

We use plum mainly for the expect(state, operator) and expect_and_grad(state, operator, options)

I think it’s only partly coincidence that this is the same thing that my PR introducing multiple dispatch in qiskit targeted.

What are the design problems and choices peculiar to Python? For example, integration with Python’s OO model.

I’m not sure what you are asking.

I came to Python from Julia rather than other way around. Julia was designed and grew with MD. But Python already has its popular paradigms or techniques. Most notably that things are organized around classes. But, maybe this is not really an issue.

the power of Julia’s dispatch system is that it works greatly with parametric types.

I definitely agree. I sort of assumed that if this exists in Python, it won’t (yet) be as solid and useful as it is in Julia.

Another thing. In Julia you often see long dispatch chains (maybe too long… unless you use a debugger to step through.) And heavy use of higher order functions used to inject behavior into code. These are usually devirtualized, inlined, elided and incur no performance penalty. But, in an interpreted language it would be difficult to design code in this style.