ipympl when jupyter and ipython kernel are in different conda environment

Hi,

Some context

I typically install the jupter notebook in a conda environment, and create a shortcut on the desktop to launch jupyter-notebook.exe (windows 10 64 bits). Then I create an ipython kernel with all dependencies I want to work with in an other environment, and I register this kernel-in-a-conda-environment with the command

ipython kernel install --user --name NameOfMyNewKernel

I find this workflow relatively clean because I can chose when I want to update either my kernel (bleeding edge or very old depending on this situation) independently from the notebook itself.

The problem

As far as I understand ipympl is really 2 packages.

• One is a python package called ipympl that must be installed together with your ipython kernel, because you need to write import ipympl in your notebook.
• The other one is a javascript package (I supposed it is called also ipympl) that must be installed together with the jupyter notebook application. I don’t know if the python package is required or not. You also need to make sure that the javascript package is registered as an extension (jupyter nbextension enable --py --sys-prefix ipympl) by jupyter.

If this is documented somewhere it is well hidden 😉

The workaround

I installed 2 times the ipympl package, one in my kernel environment, the other in the jpuyter environment. For the latter case, because the python package is also shipped in the ipympl conda package, matplotlib is installed but I think that is useless. With this, everyting works, I am able to see interactive plots in my notebook.

Before figuring out that I needed to install ipympl in the jupyter environment, I received weird error messages without really explaining what the problem was.

Proposal

1. Document the problem. I suppose that this repository README coud explain this, but I suppose it should be also documented somewhere in matplotlib and/or jupyter documentation. I must admit I find the jupyter documentation really hard to navigate though.
2. Split ipympl conda package in two: one that contains the python package and the other one containing the javascript package.

Thanks

[wholtz]

Thanks @ianhi - I was able to install the npm with:

jupyter labextension install @jupyter-widgets/jupyterlab-manager jupyter-matplotlib@0.10.4

And this did give me the desired behavior!

But…

• The documentation around how to use ipympl in a kernel is lacking. I didn’t think to try using jupyter labextension ... because all the setup instructions for jupyterlab 3+ only use pip.
• ipywidgets makes this a little nicer by having a python package (jupyterlab-widgets) for installing the javascript part.

[wholtz]

This issue is still causing people headaches. Here is Dockerfile that demonstrates the problem under JupyterLab 3:

# this is a micromamba:0.19.1 image, using digest instead of tag for reproducibility
FROM mambaorg/micromamba@sha256:1ab2666fcd1559e9865d599433671f97cb735103af9351e5e10a3f0d5b6048c8

# Create 2 conda environments:
# "lab" environment will run the jupyterlab server
# "kernel" environment will be active in the "CondaKernel" kernel within a jupyter notebook

RUN micromamba create --yes --name lab --channel conda-forge \
       jupyterlab=3.2.6 \
       jupyterlab_widgets=1.0.2 \
       python=3.10.1 \
       xorg-libxrender=0.9.10 && \
    micromamba create --yes --name kernel --channel conda-forge \
       ipympl=0.8.5 \
       ipywidgets=7.6.5 \
       matplotlib=3.5.0 \
       python=3.10.1 && \
    micromamba clean --all --yes

# activate the "kernel" environment to be active within in the next RUN command
ARG MAMBA_DOCKERFILE_ACTIVATE=1 
ENV ENV_NAME=kernel

# create a kernel.json file for using the "kernel" conda environment in a notebook
RUN python -m ipykernel install --user --name conda-kernel --display-name CondaKernel

# activate the "lab" environment for when the CMD executes
ENV ENV_NAME=lab

EXPOSE 8888
WORKDIR /home/micromamba
CMD ["jupyter", "lab", "--ip=0.0.0.0", "--ServerApp.token=''", "--ServerApp.root_dir=/home/micromamba"]

If you then open a notebook using the CondaKernel kernel, you can run:

import matplotlib.pyplot as plt
%matplotlib widget
fig = plt.figure()
plt.plot([1,2,7])

and I get

[<matplotlib.lines.Line2D at 0x7fa035b362f0>]
Error displaying widget: model not found

Our use case is that we have a JupyterHub server used by many people who use custom kernels. Currently ipympl is installed in the server environment, but user who want to make use of ipympl in their custom kernels need to exactly match the version of ipympl in the server environment. This is fragile and not expected by users. We would like to avoid installing ipympl in the server environment so that users can have more freedom to install a version of ipympl that works for them. Splitting of ipympl into two packages like ipywidgets would allow us to only install the javascript part on the server leaving the users to install the other part in their kernels.