The need for persistence

This issue is about working on making it easier to save a currently running Binder session as well as restoring/restarting a new Binder session from that state at a later point in time.

Right now when a user’s binder session times out they lose their work. This fits the “Binder sessions are ephemeral” goal but having a way to save/restore your work would be a great feature to have. In particular for public deployments like mybinder.org where the timeouts are set to fairly short times.

Below some ideas that have been discussed previously with some pros and cons. This issue is about collecting additional ideas with their pros&cons as well as zeroing in on something simple that we can implement and test drive. It “only” has to be better than what exists now to get my support (the perfect being the enemy of the good etc). We can go for more ambitious solutions in a second iteration.

Show time till timeout

The idea is to display a countdown in the UI that lets users know how long they have left till things timeout. It would also give feedback about which actions reset the timer as people would be able to see it reset. It sounds like it should be simple to implement but I don’t know if that is actually true. Can the UI access a (lower bound) on how much time is left and notice that it has been reset?

If we can access or compute how long is left this would be a nice first solution that would hopefully be simple to implement as a Jupyter notebook and Lab extension. For other UIs it would be more tricky to do.

Upload pod state to a blob store

We could execute a script via the preStop lifecycle hook of kubernetes. This script could then upload the state of the home directory (/home/jovyan) to a blob store. We’d need to find a way to tell users where to download this blob after the binder has timed out. It is also not clear that the time window the preStop script has is enough to upload everything. Unclear how a user would resume from such a download.

“Save as” uses notebook state in the browser

The state of a notebook that is open is available to the browser even after the server has gone away because the state of the notebook is only stored there. This means we could have a notebook extension that lets users save/download the notebook they are looking at even after the server has gone away. I am not sure what would need to happen or where to start. Drawbacks include that it would only cover Jupyter frontends and data files would be lost.

I will keep adding to this thread over time but please do post your own ideas and thoughts on any of these. I will try and dig out the relevant issues for the ideas that have been previously discussed so that we can pick up things from those discussions.

[manics]

Quick proof-of-concept (tested on Firefox):

1. Load a repository in binder, open a notebook, make some changes, save it.
2. Open your browser’s JavaScript console for that page
3. Paste this into the JS console and run it:

Jupyter.contents.get(Jupyter.notebook.notebook_path, {type: "notebook", content: true}).then(function(value) {
    console.log(value);
    localStorage.setItem(Jupyter.notebook.notebook_path, JSON.stringify(value));
}, function(value) { alert("Failed to get notebook"); } )

It should save your current notebook into localstorage

1. Load a new instance of the same binder repository, you must be on the same domain (if the federator directs you to a different binderhub this won’t work). Open the same notebook.
2. Open your browser’s JavaScript console for that page
3. Paste this into the JS console and run it:

loaded = JSON.parse(localStorage.getItem(Jupyter.notebook.notebook_path));
Jupyter.notebook.fromJSON(loaded)

If it works you should see your notebook with previous changes!

I think https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API would be a better option than localstorage though, it’s more complicated but it’s designed for storing much larger amounts of data.

As you’ve already mentioned the biggest problem is finding a repository identifier when storing the notebook.

[manics]

https://github.com/manics/jupyter-offlinenotebook lets you download an open notebook on any system running jupyter-notebook, it’s only the local-storage and binder links that are restricted to BinderHub, though the local storage restriction could be relaxed to work on any system.

PRs adding Jupyterlab support would be very welcome 😀