Are double percent scripts of Jupytext compatible with that of Spyder?

I see that Spyder has the practice of identifying cells based on lines starting with `# %%'. As this seems to be a common practice (among Spyder, Hydrogen, VScode) I have just implemented, in Jupytext, a Jupyter notebook converter to double percent scripts, cf. https://github.com/mwouts/jupytext/issues/59.

Could you please provide feedback on the format specifications? I found no mention of how markdown cells should be represented (or even cell metadata), so I had to made a few choices on my own, and I would like to make sure that these choices are fine with Spyder’s practice as well.

If you also want to test the converter, then install jupytext RC with

pip install jupytext==0.7.0rc0

and use the command line converter

jupytext --to py:percent notebook.ipynb            # create a notebook.py file in the double percent format
jupytext --to ipynb notebook.py                    # create a notebook.ipynb from a double percent script that has at least two cells
jupytext --from py:percent --to ipynb notebook.py  # create a notebook.ipynb from a double percent script with only one cell

[bcolsen]

@mwouts Jupytext looks like a great way to integrate the spyder text editor with the notebook plugin! I haven’t really used the jupyter notebook that much but I’ll try to be helpful there, but I have spent a couple of hours writing and thinking about this so I hope it’s helpful

I might be missing some functionality that already exists in Spyder(@ccordoba12 ?)but it would be pretty cool to execute a text file and the output goes to the notebook plugin. Then running a cell in the text editor and have it only update that notebook cell.

I my not sure what your plan is here with # %% cell separators. I’ve looked at your current plain text files and while they are quite compact and read well, if I were to write one from scratch I would have to be careful about my commenting in the code so I wouldn’t accidentally make a markdown cell. In the spirit of “Explicit is better than implicit” specifying each cell with # %% would be a good way to go.

Do you think the format is sustainable? For instance, having a special meaning for the first word after # %% (when it’s either raw or markdown) may be questionable… But I like the relative simplicity of how to create a markdown cell!

Again it would better to be explicit here as well. My take on the format would look like this:

# %% cell name [cell type] {JSON format metadata}

I looked at your examples in the issue you linked and it seems like [] aren’t part of the syntax. Here you would need to enclose the cell type in [] so there is no ambiguity between the name and the type. The order wouldn’t actually matter, everything could be optional, and it dosen’t rely on special words. This way one day you could have a crazy file with [R], [julia], [python], [html], and [latex] blocks or some craziness. If you just want a markdown block its still just # %% [markdown] not too long or complicated.

Even though the order wouldn’t matter, I’m suggesting name first for compatibility with existing editors that use the labels like Spyder does in the outline explorer. The editors will still likely take the whole line as the name but at least it will start with something user defined rather then seeing a bunch of [markdowns].

As for the non-code cells I would suggest wrapping them in """ strings this would be similar to the php heredoc style. This could allow for an editor to change the syntax highlighting to markdown like many do for html in php scripts. Perhaps more excitingly it would also allow f-stings and sting formatting in markdown. But this might be challenging to get back from the notebook to a text file.

Spyder recently implemented grouped code cells with the # %% as the top cell and # %%% as a subgroup. You could use this to label code cells with the appropriate heading styles.

[mwouts]

I would say that this conversation has filled it purpose in that jupytext now supports Spyder style “percent” cells.

I do agree 100%. I think we’re done with this question and that now, the Spyder format and the py:percent format in Jupytext do match well (so I’ll close this issue, thanks again for your feedback).

Nice idea, I opened an issue in the notebook plugin.

Thank you @jitseniesen !

Indeed, it would be great to offer integration directly in Spyder, like is done in VIM with the jupytext.vim plugin: when the users open an .ipynb file in the editor, they get the .py form, and when they save it, the .ipynb file is updated (re. the other way around, i.e. opening .py files as notebooks in Jupyter, this does come with Jupytext and its custom contents manager).