Some matplotlib backends on MacOS significantly slow down jupyter

On my MacOS Mojave system, enabling some matplotlib backends will significantly slow down the execution of subsequent cells (the slowdown starts to occur about half a minute after setting the backend). See the screenshot below for an example (notice the 10.0s execution time of the third cell):

In particular, I notice that the 10.0s consists of three “stages”:

1. After I tell jupyter to run the cell, the kernel stays idle for a few seconds;
2. The kernel becomes busy, and the output immediately appear;
3. The kernel stays busy for another few seconds after the output appears.

I’ve tested with different matplotlib backends, and the issue happens when using some of the backends. Here is a summary:

• Backends that make this issue happen:
  • osx
  • qt
• Backends that do NOT make this issue happen:
  • inline
  • agg
  • tk
  • nbagg
  • notebook

I’ve also tested with different ipython/jupyter clients, and the issue happens when using some of the clients. Here is a summary:

• Clients where this issue happens:
  • jupyter notebook
  • jupyter console
  • jupyter qtconsole
  • jupyter lab
  • Nteract app
• Clients where this issue does NOT happen:
  • ipython console

When tested in jupyter console, an additional warning is cast when executing the third line in the screenshot above:

/usr/local/lib/python3.7/site-packages/jupyter_console/ptshell.py:661: UserWarning: The kernel did not respond to an is_complete_request. Setting `use_kernel_is_complete` to False.
  warn('The kernel did not respond to an is_complete_request. '

I have tried to dig into the source code of IPython.core.pylabtools.activate_matplotlib, and here is a minimal (to my best) example:

$ jupyter console
Jupyter console 6.0.0

Python 3.7.2 (default, Jan 13 2019, 12:50:01)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.2.0 -- An enhanced Interactive Python. Type '?' for help.



In [1]: import matplotlib

In [2]: matplotlib.rcParams['backend'] = 'MacOSX'

In [3]: import matplotlib.pyplot

In [4]: import time

In [5]: time.sleep(35)

In [6]: 'test'    # this line will take a lot of time
/usr/local/lib/python3.7/site-packages/jupyter_console/ptshell.py:661: UserWarning: The kernel did not respond to an is_complete_request. Setting `use_kernel_is_complete` to False.
  warn('The kernel did not respond to an is_complete_request. '
Out[6]: 'test'

My system information that might be relevant:

$ python3 -c "import IPython; print(IPython.sys_info())"
{'commit_hash': '523ed2fe5',
 'commit_source': 'installation',
 'default_encoding': 'UTF-8',
 'ipython_path': '/usr/local/lib/python3.7/site-packages/IPython',
 'ipython_version': '7.2.0',
 'os_name': 'posix',
 'platform': 'Darwin-18.2.0-x86_64-i386-64bit',
 'sys_executable': '/usr/local/opt/python/bin/python3.7',
 'sys_platform': 'darwin',
 'sys_version': '3.7.2 (default, Jan 13 2019, 12:50:01) \n'
                '[Clang 10.0.0 (clang-1000.11.45.5)]'}

$ pip3 freeze | grep 'jupyter\|matplotlib'
jupyter==1.0.0
jupyter-client==5.2.4
jupyter-console==6.0.0
jupyter-contrib-core==0.3.3
jupyter-contrib-nbextensions==0.5.1
jupyter-core==4.4.0
jupyter-highlight-selected-word==0.2.0
jupyter-latex-envs==1.4.6
jupyter-nbextensions-configurator==0.4.1
matplotlib==3.0.2

Thanks!

[impact27]

I think this is caused by AppNap: https://developer.apple.com/library/archive/documentation/Performance/Conceptual/power_efficiency_guidelines_osx/AppNap.html This can be desactivated by https://pypi.org/project/appnope/

[ccordoba12]

Ok, closing then.