Create MagicUI based UI using on a JSON or similiar

Hi,

imagine you have a JSON file, that is used to define the input elements for an UI (example below). What would be the best approach to get started to create a magicgui-based interface using this as an input? or in other words, is there an example etc., where can I get an idea how to “dynamically” create an UI based on a JSON or dict etc?

The idea is to (maybe) create a Napari plugin that can run our APEER modules, which come with such an Input UI specification. In our ZEN software and on www.apeer.com we already use such specs to create the UI dynamically and I wonder if this would work using magicgui (inside Napari) as well?

We already a commandline-based APEER module executor, which is completely docker-based and can in principle run on Windows on Linux.

Any ideas are appreciated?

{
    "spec": {
        "inputs": {
            "input_image": {
                "type:file": {}
            },
            "red": {
                "type:number": {
                    "lower_inclusive": 0.0,
                    "upper_inclusive": 1.0
                }
            },
            "green": {
                "type:number": {
                    "lower_inclusive": 0.0,
                    "upper_inclusive": 1.0
                }
            },
            "blue": {
                "type:number": {
                    "lower_inclusive": 0.0,
                    "upper_inclusive": 1.0
                }
            }
        },
        "outputs": {
            "tinted_image": {
                "type:file": {}
            }
        }
    },
    "ui": {
        "inputs": {
            "red": {
                "index": 51,
                "label": "Red channel",
                "widget:slider": {
                    "step": 0.1
                }
            },
            "green": {
                "index": 52,
                "label": "Green channel",
                "widget:slider": {
                    "step": 0.1
                }
            },
            "blue": {
                "index": 53,
                "label": "Blue channel",
                "widget:slider": {
                    "step": 0.1
                }
            }
        },
        "outputs": {}
    }
}

This renders in our SW like this:

[tlambert03]

in the first take APEER modules take files as an input and produce files as an output as well. So maybe we can “workaround” subprocesses in the beginning … 😃

ok, that does simplify things to start with! so, a basic first implementation might look something like this:

1. manually start up an apeer docker server (assuming there’s something like a server?)
2. in another process, start up napari.
3. in your napari widget, you’d have some interactive element (such as that generated by magicgui) that would either ping your server notifying it of some workflow, or, I guess, drop a file into a folder that the server is watching?
4. apeer does its business
5. then you need something back in napari to get notified when a result is ready. This could be something like folder “watcher” running in another thread (something like watchdog, or we can roll our own type thing). And then that watcher needs to emit an event “file ready” that is connected to some callback in your widget that reads the file and adds it to the viewer.

there’s obviously a lot of important details left out there 😂 but that’s what a basic two-process file-based communication scheme could start out as

[tlambert03]

yep, all makes sense. would be an awesome use case. I haven’t looked into apeer or the module system, but provided that you can get that JSON string in your first post, it would be feasible to generate a UI for it (some more thoughts on that below). As for napari–>docker APEER–>napari, that also seems feasible, but of course, much more involved than the GUI generation… so let’s leave that discussion for a future date 😃

If you want to get started playing with this now, the challenge is in converting the schema from your ui json to python types that magicgui can recognize. So basically, you need to convert this:

{
            "red": {
                "type:number": {
                    "lower_inclusive": 0.0,
                    "upper_inclusive": 1.0
                }
            },

            "red": {
                "index": 51,
                "label": "Red channel",
                "widget:slider": {
                    "step": 0.1
                }
            }
}

into this:

from magicgui import widgets

red_slider = widgets.FloatSlider(min=0, max=1, step=0.1, label='Red Channel')

or, probably more useful in this case is the magicgui.widgets.create_widget factory function:

red_slider = widgets.create_widget(
    annotation=float,
    widget_type='FloatSlider',
    label='Red Channel',
    options={'min': 0, 'max': 1, 'step': 0.1}
)

to get things playing nicely in napari, you can use a type of napari.types.ImageData for your input_image field (which, once the widget is in a napari viewer, will render as a dropdown menu showing the available image layers).

from napari.types import ImageData

image_dropdown = create_widget(annotation=ImageData, label = 'Input Image')

you can then combine these subwidgets into a container:

container = widgets.Container(widgets=[red_slider, image_dropdown, ...])

# and add that to napari:

viewer.window.add_dock_widget(container)

This doesn’t cover what events happen when the user interacts with your GUI… but it’s a start. See https://napari.org/guides/stable/magicgui.html for more details on napari-specific types that you can use in magicgui (like ImageData), and see the magicgui widget overview and associated docs for more details on the fields accepted by magicgui.

But again: all of this should be eventually done automatically for you!