Proposal: Embedding Obseverable Notebooks in Streamlit

This is a proposal for adding st.observable to Streamlit. You would not only be able to embed Observable notebooks into a Streamlit app, but also inject your own data from Python into the Observable notebook, and use the Observable notebook as a widget in the Streamlit app.

Problem

• As a user, I want to use custom elements that I could only currently do with unsafe_allow_html.
• As a user, I want the option to have complete control of the charts and graphics in my Streamlit app, outside of the current builtin charts and integrations.
• (stretch) As a user, I want to be able to create my own custom widgets (forms, sliders, pickers, etc.) with my own interactions and user interface.

Solution

(Quick Observable notebook review - every notebook is composed of cells, and each cell either has a unique name or is anonymous. The cells are built into a DAG and executed only when needed)

With this proposal, there will be 3 main use cases for embedding Observable notebooks in a Streamlit app:

1. Embedding plain Observable notebooks with no modifications

st.observable("@tmcw/hello-world")

Examples:

• Showing a tweet
• Display a small art piece
• Show an elk graph
• Have a custom explainer for a complex topic (ex2).

2. Embedding Observable notebooks and injecting your own data from Python

st.observable("@tmcw/hello-world", redefine={"name": "Streamlit"})

Examples:

• Inject your own location data into a voronoi map,
• Display a more comprehensive view of a neural network or of a handwriting classifier
• inject your ML training results into a custom data viz
• Create dynamic earthquake intensity graphs
• Create custom planet textures
• Display force graphs of data from Python

3. (stretch) Using an Observable notebook as a custom widget

notebook = st.observable("@tmcw/hello-world")
name = notebook.observe("name")
st.write("Name is", name)

Examples:

• Allow users to draw on a canvas and use it in Python
• Create your own HTML form with any inputs and custom CSS
• Matrix inputs
• @jashkenas/inputs for color pickers, map coordinates pickers, local file chooser, passwords, etc.

Potential Code Samples

import streamlit as st
    
## embed the "hello, world" notebook in full into Streamlit app
st.observable("@tmcw/hello-world")
    
## Embed only the "viewof gl" cell from mbostock's image-warping notebook
st.observable("@mbostock/image-warping", targets=["viewof gl"])
    
    
# using your own geo data, inject into a chart of the US and add a voronoi diagram
geo_data = pd.read_json("https://...")
st.observable("@mbostock/u-s-airports-voronoi", targets=["chart"], redefine={
  "data": geo_data
})
    
# Observe value of the "object" cell and use it as a widget
form = st.observable("@mbostock/form-input", targets=["viewof object"])
    
obj = form.observe("object")
    
st.write("Message: ", obj.get("message"), "Hue: ", obj.get("hue"))

st.observable Definition

    def observable(
            self,
            element,
            notebook,
            targets=None,
            redefine=None,
            iframe_src="https://onb-host.glitch.me/notebook.html" # Or some Streamlit-specific host 
            ):
            """Display an Observable notebook.
            Parameters
            ----------
            notebook : str
                The identifier for the ObservableHQ notebook to render, 
                or a link to a JS ES module to define a notebook.
            targets : list of strings or None
                List of cell names to render (in the order of the list).
                If not provided, all cells are rendered.
            redefine: dict or None
                The keys are the name of the cells to redefine,
                values are the JSON serializable value of the cell
                you want to refine
            iframe_src: str or None
                An alternative src for the Observable notebook iframe. 
        """

Things to Consider (security, required setup, etc.)

• Since Observable notebooks are basically just running arbitrary Javascript, each notebook embedding should be in an iframe. This won’t solve all security problems, but it’ll at least sandbox the notebook away from the Streamlit window/DOM.
• The iframe should ideally have a different origin, for security purposes. Maybe Streamlit could invest in a domain like [observable.streamlit.io](http://observable.streamlit.io) (that could be configured to a self hosted option, like st.observable("@/b", iframe_src="observable.my-company.com/notebook.html")).
• There would be security concerns when you redefine a cell in a notebook - you’re passing data directly into arbitrary Javascript and 3rd party packages, so Streamlit authors should at least be aware (and know how to audit their notebooks).
• Since Observable notebooks are compiled to ES modules, the Streamlit React app would need to support dynamic imports (or have a polyfill)
• You could only observe or redefine with JSON serialize data structures (dicts, list, str, etc.). This is because I don’t know of another dynamic solution to pass data from Python/Javascript other than json.dumps in Python and JSON.parse in JS. Maybe there could be a way to pass around buffers or raw bytes (observing webcam data, pass image/audio data from python to observable notebook, etc.), but that could be figured out later.
• Observable’s terms of service by default doesn’t allow for using Observable notebooks outside of Observable (unless the author allows it or the author licenses the notebook differently). There’s no real easy way to check for this yet, but there’s been some discussion and I believe that this will be resolved soon.
• If observer/widget support is made, st.observable, fulfills nearly all front-end plugin use cases that I could think of. If someone wants to use some specific charting library, they can just make it in Observable and import the notebook into Streamlit.
• Observable notebooks can be difficult to learn as a beginner (especially for Python-only people). Outside of general Javascript, you’d have to learn some Observable-specific syntax/runtime quirks before you could build your own customized notebooks.
• 3rd party Observable extensions haven’t really been built in popular products yet. It’s certainly possible, but afaik there’s no jupyter/r shiny/rstudio Observable integrations. I think that’s just because Observable is still fairly new, and not because of any inherent flaws with Observable.
• Maybe Streamlit could have an ObservableHQ team and host notebooks for easy access to common needs? e.g. @streamlit/webcam, @streamlit/google-photos-chooser, @streamlit/form, etc.

Needs more exploring

I’m not sure how the observe python API should work, since it doesn’t seem very Streamlit. I feel like the return value of st.observable should be similar to st.text or st.line_chart where it can be later re-defined to another Streamlit element, like this:

    a = st.observable('@tmcw/hello-world')
    
    if st.button("Override the above text"):
        a.text("This is now text!")

But, if we want to be able to observe values of an embed notebook, then it also makes sense for it to act like a widget like slider or text_input :

    t = st.text_input("Name:")
    s = st.slider("Age:")
    o = st.observable("@fake/phone-number-input", targets=[])
    
    phone = o.observe("phone_number")
    
    t.text("new") # not allowed, t is int
    s.text("new") # now allowed, s is str
    o.text() # Also shouldn't be allowed, maybe? Since o.observe was called
    

I think this is hard because, in this proposal, an Observable notebook can be normal Streamlit element like line_chart or text, but can also be a Streamlit widget like slider or text_input.

Resources

• “🤔 How to: Embed a Notebook in a React App” from ObservableHQ
• “A React component for rendering Observable Notebooks” from me
• “Observable Notebook iframe Protocol” from me - defining how a host can create and interact with an Observable notebook in an iframe (WIP)
• “react-observable-iframe” from me, implementation of the above protocol (also a WIP). Something like this would have to be built in /frontend

---

Note: this is something I’d love to work on! I’ve made a lot of tooling/packages around Observable notebooks, and I’d love better integration with then in Streamlit (and I need some PRs for Hacktoberfest).

[asg017]

Update: I wrote a custom component for rendering Observable notebooks called streamlit-observable. You can check out an example app here: http://streamlit-observable.herokuapp.com/

[asg017]

Aw sweet, looks like that would work! I’ll close this and work with that (and if st.iframe doesn’t work out, I’ll just re-open this. Thank you!