Syncing local python project files with remote Jupyter server

Feature: Notebook Editor, Interactive Window, Python Editor cells

Description

Microsoft Data Science for VS Code Engineering Team: @rchiodo, @IanMatthewHuff, @DavidKutu, @DonJayamanne, @greazer

Context

VSCode allows users to connect with a running remote Jupyter server (https://code.visualstudio.com/docs/python/jupyter-support#_connect-to-a-remote-jupyter-server). Using the Jupyter API it is able to start a kernel and execute cells of a locally saved notebook remotely.

Use case/problem

If you want to for example call a function from another python file (foo.py) from that notebook (local-notebook.ipynb), the remote kernel can’t access local files. The remote kernel can access other files saved on the remote notebook server. The problem is that local (Python) files are not synced with the remote Jupyter server instance, for the remote Python interpreter (kernel) to access them.

Existing solutions

The standard way of achieving this is through rsync over SSH. However this default requires managing a SSH connection and SSH keys (which large entreprises servers not necessarily allow). There are workarounds (manually uploading files through the notebook UI, and using git), but these inhibit development and iteration speed.

Proposal

Extend the VSCode Python extension to allow users to sync files with a remote running Jupyter notebook server. Under the hood, the Jupyter contents API can be used for this:

• https://jupyter-notebook.readthedocs.io/en/stable/extending/contents.html,
• https://github.com/jupyter/jupyter/wiki/Jupyter-Notebook-Server-API.

Authentication and authorization is handled through the API token that you need anyway to connect.

No SSH, git or manual hassle required.

Additionally, you can execute your local code (by calling it through the notebook) remotely without having to manage a remote Python SSH interpreter, or docker images. All you need is a running jupyter notebook.

[IanMatthewHuff]

@brunocous Thanks for the detailed suggestion. I do feel that this would be an interesting suggestion to consider. We’ll discuss it at our triage meeting.

[rchiodo]

@alfredodeza thanks for the upvote. There’s no recommended way to sync files other than for you to put them in the same folder where you started the remote server (that will make the relative paths work correctly).