Render stored Matplotlib png output as a fallback when opening a notebook

Actual behaviour

With the current ipympl and using %matplotlib widget as a backend VS code can render this example without issues:

%matplotlib widget

from matplotlib import pyplot as plt

fig, ax = plt.subplots()
ax.plot([0,1,2], [2,3,4] ,"o-")

Though, when saving and reloading the notebook file no image is shown (although a static image is available):

Expected behaviour

The small </> icon on the left allows to change the mimetype representation. When this is changed manually to png the alternative widget representation that was previously saved is shown correctly:

So VS code does not recognize that without a running/current widget available that the png static representation should be shown instead.

Environment data

• VS Code version: 1.63.2
• Jupyter Extension version v2021.11.1001550889
• Python Extension version v2021.12.1559732655
• OS (Windows | Mac | Linux distro) and version: Windows 10
• Python and/or Anaconda version: 3.9.9
• Type of virtual environment used (N/A | venv | virtualenv | conda | …): conda
• Jupyter server running: Local

[mjbvz]

@rebornix To support this, we need to transfer the entire output to the webview. Today I believe we just send over the output for the selected item mime type

That would allow us to fallback to a different renderer if we can’t find one for the current mime type

Let’s look into this for October

[rchiodo]

Thanks for the bug. VS code doesn’t allow for a fallback mimetype. On reload it just uses the widget data as that’s what was selected before.

We don’t support reloading widgets at the moment (see this bug for more info: https://github.com/microsoft/vscode-jupyter/issues/5424#issuecomment-815065089)