Inconsistencies, rolling back edits, and keeping track of the document's global state

Hi,

I’m looking at the kernel development docs to get a sense of how hard it would be to support proof assistants like Coq, Lean, or F*.

One feature that these languages have in common (unlike, say, Python or F#), is that they are able to roll-back changes to the environment. That is, the typical way in which you interact with a Coq or F* document is that you send small chunks of the document, one by one, to the theorem prover / language server. Every time you edit a previously-processed chunk, the editor sends a message to the prover to tell it to rollback that particular section of the document, ensuring that the prover’s internal state is always in sync with the contents of the document.

In other words, if I write let x = 1 and send that to F*, then change my mind and rename x to y, then the old binding for x does not persist (I only get y).

This seems at odds with Jupyter’s model, in which processing a definition x = 1 in Python, then renaming x to y and resending the definition (now y = 1) causes two bindings to coexist (x and y). That model can lead to inconsistencies, of course, where I rename a function after processing other definitions that depend on it and forget to update the other definitions. This will work until I restart jupyter, at which point the incorrect definitions will stop working.

This shadowing without rolling back is not a big problem in Python, but in C++ for example it is an issue: I tried running the cling-zeus kernel advertised at https://blog.jupyter.org/interactive-workflows-for-c-with-jupyter-fe9b54227d92, for example, and first wrote void test() { return; } before changing it to int test() { return 1; }. That change was rejected with this following message:

input_line_8:1:5: error: functions that differ only in their return type cannot be overloaded
int test() {
~~~ ^
input_line_7:1:6: note: previous definition is here
void test() {
~~~~ ^

So, the TLDR:

• Can kernel request that code cells be sent in order? That is, can I write a kernel that requires UI to execute cells 0, 1, and 2 before executing cell 3? The closest I’ve found is https://github.com/stitchfix/nodebook (linked from https://www.reddit.com/r/datascience/comments/7grh1j/when_should_jupyter_notebooks_be_avoided/dqlakqy/)
• Can kernels know where in the document the cell that’s being executed is? Or do they only see the contents of the cell?
• Does the protocol give kernels a way to get notifications when a previously-executed cell is edited? For example, if I execute cells 1, 2, and 3, then edit cell 2, can the kernel subscribe to a notification saying “cell 2 was modified”?

Thanks for Jupyter, btw — it’s great 😃

[cpitclaudel]

I assume you’ve already read the Jupyter kernel docs?

I have 😃 Or more precisely, I did when I posted the original issue. I don’t remember everything right now 😃

Specifically, what additional messages are needed?

Roughly, what’s needed is some way for the prover to understand the current state of the document. There are many ways to achieve this: sending all cells instead of the latest one (letting the prover compute the difference), or giving an identifier to each cell and including the id of the previous cell when sending each new cell.

UIs would also need to be able to display the status of a cell (sent or unsent); and, if a user try to send a cell, the UI should amke sure to send all preceding unsent cells, too.

I took a look at jscoq + hott (rhino-hott) the other day and was impressed. The IJavascript kernel would also still need additional kernel messages AFAIU.

You may be curious to try https://people.csail.mit.edu/cpitcla/fstar.js/stlc.html , too 😃

[westurner]

https://github.com/EugeneLoy/coq_jupyter :

Backtracking There are number of convenience improvements over standard Jupyter notebook behaviour that are implemented to support Coq-specific use cases.

By default, running cell will rollback any code that was executed in that cell before. If needed, this can be disabled on a per-cell basis (using Auto rollback checkbox).

Manual cell rollback is also available using Rollback cell button (at the bottom of executed cell) or shortcut (Ctrl+Backspace).

• ENH: ipykernel, : similar functionality
  • https://django-reversion.readthedocs.io/en/stable/api.html#django-reversion-api
  • https://pyrsistent.readthedocs.io/en/latest/api.html#pyrsistent.PMap.evolver
    • PMap doesn’t do rollback or git diff dict() dict(), fwiu
  • https://github.com/jupyterlab/jupyterlab/issues/1639#issuecomment-925223852

    jupyterlab/rtc synchronizes with a y.js CRDT for the one notebook but not the whole IDE?

  • IIRC the effective per-cell locals and globals are set by ipykernel