VTK Modules and downstream packages as Plugins

After chatting with @CharlesGueunet, we are wondering if we could develop a mechanism in PyVista for registering different VTK filters to DataSetFilters at runtime. This would be particularly useful for a few different use cases including:

1. When VTK is built with a non-default module enabled that adds some new vtkFilterClass for a datatype. We should avoid having try: import vtkFilterClass from vtkmodules all over the place and a crowded namespace from filters that may or may not be enabled.
2. 3rd Party libraries like tetgen that have restrive licenses. Tetgen is essentially just a PolyData filter and it would be great to have a mesh.tetgen.tetrahedralize() filter on PolyData objects.

I am thinking that the best way to handle this would be through a mechanism that adds a namespace to the right dataset filters classes at runtime. To implement this, we could have third-party packages define an entry_point that registers their filter methods with PyVista for different dataset filter types.

In PyVista, we could provide a base mixin class to handle common functionality:

class DataSetPlugin:
    """Base class to define methods that should be available for all PyVista data types."""
    ...

class PolyDataPlugin:
    """Base class to define methods that should be available only for PolyData types."""
    ...

In the downstream package, a mixin class would be implemented from this base like:

class TetGenPlugin(PolyDataPlugin):
    """The TetGen module as a set of PolyData filters."""
    def tetrahedralize(self, *args, **kwargs):
        tet = tetgen.TetGen(self)
        tet.tetrahedralize(*args, **kwargs)
        return tet.grid

Then the third-party package, they would register each of their classes as entry points like:

setup(
    ...
    entry_points={'pyvista.plugin': ['tetgen = tetgen:TetGenPlugin']},
)

Then in PyVista, we would have logic at runtime (on import) to iterate over the entry points and register them with each filter class appropriately. This would use setattr to create a subnamespace on DataSetFilters, PolyDataFilters, etc. for each registered class. In the example above, we would register the tetgen plugins under a tetgen namespace such that any filters defined in that plugin can be accessed like:

mesh = pv.Sphere()
grid = mesh.tetgen.tetrahedralize()

---

Then any VTK module could ship as a built VTK wheel and a third-party package. Personally, I’d like to follow a naming scheme of pyvista-<plugin> when possible so that way a user would simply have to do:

$ pip install custom_vtk.whl pyvista-plugin
$ python
>>> import pyvista
>>> ...
>>> mesh.plugin.filter()
...

---

I’d love some thoughts and suggestions on other ways to implement this, @pyvista/developers

[banesullivan]

1. In the case of TetGen, PyVista cannot rely on TetGen due to its GPL license - anything as simple as import tetgen in PyVista would require adopting the GPL license. TetGen is truly a downstream package, relying on PyVista. If we can add the ability for PyVista to load any plugins through entry points, then we can avoid license headaches like this

2. Yes! Totally agnostic. The goal is for downstream packages to register their entry points and PyVista will iterate over the entry points to see what Plugins it should load/register. This will be done by iterating over pkg_resources.iter_entry_points to fetch the plugin classes, then mix them in with setattr. So in this case, all of the plugins would have to be downstream from PyVsita.

3. I believe with the right type annotations this can still work in IDEs for auto-complete though it requires some special sauce to work and won’t always be available to everyone. You would have to do something like:

class PolyDataTetGen(PolyData):
    tetgen = TetGenPlugin

...

poly: PolyDataTetGen = pv.Sphere()
poly.tetgen...

---

I’d like to give this some more thought and see if there are other, better approaches to this problem. The same scenario has been implemented time and time again across Python, e.g. xarray’s accessors being one example to follow: https://xarray.pydata.org/en/stable/internals/extending-xarray.html

[CharlesGueunet]

@banesullivan any news on this ? how can I help ?