Discuss: File format in the .NET Interactive VS Code extension

The primary reason for the experimental file format supported by .NET Interactive Notebooks is that there are a few design decisions to be worked out to enable the Microsoft Python extension for Visual Studio Code and the .NET Interactive Notebooks extension to work well together. Until a design is agreed upon, we want to avoid introducing potential compatibility issues into .ipynb.

Here are some of the challenges:

• There is not an agreed-upon convention for polyglot notebooks in the .ipynb file format that also takes into account:
  • single-process polyglot kernels, and
  • multiple kernels (whether polyglot or not) within a single notebook.
• There isn’t currently a clear way to indicate which VS Code extension should handle a given .ipynb file, and Python clearly takes precedence.

---

I’d like to get people’s thoughts on the experimental .dib file format (example below) that the .NET Interactive Visual Studio Code extension supports.

.ipynb is also supported, as well as conversion between the two.

These are the goals we’ve had in mind that led to the current state of the design:

• It should be trivial to author and read in a simple text editor without special tooling and without concern about escaping, e.g. not JSON, XML, etc.

• It should be copy-pasteable between different contexts, e.g. a bare script file, a notebook cell, or another .NET Interactive host (e.g. https://github.com/CodeConversations/CodeConversations)

• It should be able to contain multiple languages, including languages not yet supported by .NET Interactive.

• It should be amenable to a good tooling experience, including completions and other language services even when multiple languages are present.

• It should be able to be opened as a notebook (as currently implemented by the .NET Interactive VS Code extension).

• It should be executable as an automation script. In the simple case this need not be a polyglot script, but it should be able to take advantage of the full capabilities of .NET Interactive, e.g. magic commands, NuGet support, extensibility, variable sharing.

  • When in automated mode, it should be able to advertise and consume command line arguments in a readable, simple-to-author manner.

  • When in interactive mode (e.g. notebook, REPL), the command line inputs should be able to be collected from the user via a prompt or trivially configured inline the code.

Here’s a simple example of the format:

#!powershell

Write-Output "Hello from PowerShell!"

#!fsharp

let hello = "Hello from F#"
hello |> Console.WriteLine

#!csharp

#r "nuget:Humanizer,2.8.1"
using Humanizer;

#!share --from fsharp hello

Console.WriteLine(hello.Replace("F#", "C#").Transform(To.TitleCase));

[jonsequitur]

If dib is the file format supported in VSCode, is it only limited to .Net languages?

.dib isn’t language-specific and other languages can be added dynamically by adding subkernels.

If it’s intended to be more widely adopted, is this file format extendable and easily convertible so that other language communities can extend their notebook experience in VSCode?

The file format can be extended to the degree that .NET Interactive itself can be extended, whether programmatically within the script or via a NuGet extension.

[jhoneill]

I’ve come to this as a PowerShell person who wants to author PowerShell notebooks, preferably with VS code.

• .NET interactive lets me create an ipynb notebook through Jupyter, using a local web server, a stack of python bits and so on. The learning curve is not too bad, but the end product is kind of ugly.
• ipynb files are opened with the Python extension which works up to a point, but adding a new cell (for example) fails if the notebook uses a .NET langauge. And is also kind of ugly.
• DIB files have a glaring omission which inhibits their use - not saving results. (a work round is to save as ipynb)

The current experience is you can Open a notebook and Run some code which generates output; Closing the notebook will prompt to Save changes, but later Re-opening the notebook shows the changes were NOT saved. That’s referenced in #500 and it’s the difference between an interesting proof of concept and something which can be used in the real world.

I’m not clear what the objective for DIB files is. Avoid competing with the Python extension for ownership of ipynb files ? Using the same format with a different extension would do that. An experimental way of creating polygot files ? You have solved that for ipnyb export by telling Jupyter everything is c# and using the language magic commands for each block of code. The top of the list of aims you have

“It should be trivial to author and read in a simple text editor without special tooling and without concern about escaping, e.g. not JSON, XML, etc.”

And that’s not an aim I’d normally take issue with. But this is a format for doing notebooks in VSCode so “edit anywhere” is not the priority it might be for other types. JSON, XML etc lend themselves to machine parsing, (type .\test.ipynb | ConvertFrom-Json).cells gives access to the source, outputs, etc. Given that files will be edited almost exclusively in VSCode - at least in the short term, ease of building a notebook as the output of one process, or reading it as the input to another, should carry more weight than ease of hand editing the files.