Hookspecs for layer keybindings

🚀 Feature

Plugin developers are beginning to use our reader/ writer hookspecs with success, but are looking to add additional functionality like custom key bindings, mouse functions, or GUI elements (see https://github.com/hci-unihd/covid-if-annotations/issues/3).

I propose that the next hookspecs we add are for our key bindings for our layers. We can then grow to other hookspecs after.

Motivation

I think we should continue with our functional approach, and our approach of not providing “napari” objects to plugins but instead providing things like layer data tuples, e.g. (data, meta, layer_type). The advantage of this approach is that it enforces a “loose coupling” with napari, our plugins can’t just do anything like nuke the whole viewer or layer, and it gives us a lot of control over what a plugin does as napari calls the plugin code. The downsides are some reductions in flexibility, and potentially performance, but I think they are well worth it, if they are even really there.

Pitch

It’s unclear if hookspecs are the natural thing for what I have in mind, but I think each plugin should be able to provide one dictionary per layer type. The keys of that dictionary need to valid key press combinations for napari, and the values of that dictionary need to be functions with the following signature:

image_key_bindings_func(data: Any, meta: dict) -> (Any, dict)

Note that this is different from the API of our current keybindings functions which take in a layer and can then do anything. We could keep that API around or eventually deprecate it for this one.

How these keybindings would work is that when the key was pressed we’d generate the (data, meta) tuple pass those to the function, and then get back a (new_data, new_meta) tuple. If new_data is not None we’d set layer.data=new_data, and then we’d go through every element of new_meta and update the layer property accordingly. This is a massive reduction in what keybindings can do, but I think it will actually make them much easier to understand and make this whole system more robust.

One question would be do we “copy” the data when we pass it to the keybindings function, I’m not sure if we’d want to do that as it might trigger some big delays, but it might make the whole system much more “safe” as no in place modifications would be possible.

Curious what do @jni @tlambert03 @kne42 think of this approach? and how to best integrate it with the napari_plugin_engine? We could add a special “hookspec” where people are able to provide dictionaries instead of functions.

An interesting exercise would be how many of our current layer keybindings could be rewritten to conform to this API? Unfortunately not all of them as we have keybindings do things like change mode (like painting/ pan_zoom) amongst other things, which aren’t inside our meta dict, but maybe we could figure out how to fix that.

[ttung]

I agree with lots of ideas that have been stated. Due to an abundance of laziness, I won’t attribute the idea to the original author (sorry!), but here’s my random thoughts:

1. It would seem like keybindings are specific enough to the application that it is necessary to pass in napari-specific objects to the plugin. The idea of proxy objects is interesting, as it significantly reduces the exposed surface and gives us something like a fixed API that we can maintain. However, nothing really stops the plugin developer from import napari and going to town. If you want a constrained plugin interface with good isolation, python is probably the wrong language[1].
2. We need to make data futures a first class concept in Napari. It is absolutely required for managing large data sets. It might even be prudent to force all plugins to use data futures, even if napari internals isn’t ready for it.
3. Making the plugin interface use keyword-only arguments allows for much flexibility for API changes. On the other hand, if such flexibility is not desired, then please disregard this point.
4. Meta-question: Why are plugins making keybindings? I can’t think of any plugins that set keybindings, and even if they are making keybindings, I would imagine that they ought to be doing it on an optional basis, and through an existing config interface. “Thanks for installing Plugin XYZ™. Would you like to add the default keybindings?” -> [Yes/No] -> “The default keybindings for Plugin XYZ™ conflict with your existing keybindings.” -> [Don’t set keybindings/Override keybindings/Review keybindings].

[1] It would be super fascinating to explore a plugin interface based on python multiprocessing + shared memory, as that would deliver the constrained plugin interface with good isolation.

[tlambert03]

and I’ll add that I’m definitely not totally opposed to the pattern you just proposed there 😄

I certainly agree there are code safety concerns, and it would be nice to be able to prevent something like this:

@napari_hook_implementation
def napari_add_keybinding():
    return [('labels', 'Space', lambda layer: del layer)]

at the same time, there’a a good percentage of our current keybindings that we wouldn’t be able to implement with this pattern.

which gets me thinking, what about a proxy object? Something that behaves very much like the original object API, passing through “approved” methods and attributes, but preventing things like deletion, or any other unapproved methods…