[BUG] Spurious "extra unexpected referrers" warning when using python debugger

ALL software version info (bokeh, python, notebook, OS, browser, any other relevant packages

bokeh 1.2.0

panel 0.6.0, python 3.6.7, Mac OSX 10.14.3, PyCharm 2019.1

Description of expected behavior and the observed behavior

Ran a bokeh application in debugger from inside of PyCharm. After closing application Window in browser, got seemingly spurious warning about module having “extra unexpected referrers”.

This does not occur if not run with debugger.

Complete, minimal, self-contained example code that reproduces the issue

Here is the script:

import panel as pn
from bokeh.command.util import build_single_handler_application
from bokeh.server.server import Server

if __name__ == '__main__':
    server = Server(build_single_handler_application(__file__))
    server.start()
    server.show('/')
    server.io_loop.start()
else:
    pn.Pane('hi').servable()

Stack traceback and/or browser JavaScript console output

ERROR:bokeh.document.document:Module <module ‘bk_script_1001’ from ‘/Users/cbarber/ws/fabml/panel_apps/bug.py’> has extra unexpected referrers! This could indicate a serious memory leak. Extra referrers: [{‘self’: <bokeh.document.document.Document object at 0x114814400>, ‘get_referrers’: <built-in function get_referrers>, ‘FrameType’: <class ‘frame’>, ‘module’: <module ‘bk_script_1001’ from ‘/Users/cbarber/ws/fabml/panel_apps/bug.py’>, ‘referrers’: […]}, {‘self’: <bokeh.application.handlers.script.ScriptHandler object at 0x10181cac8>, ‘doc’: <bokeh.document.document.Document object at 0x114814400>, ‘module’: <module ‘bk_script_1001’ from ‘/Users/cbarber/ws/fabml/panel_apps/bug.py’>, ‘old_doc’: <bokeh.document.document.Document object at 0x1148149b0>, ‘old_io’: {‘output_notebook’: <function output_notebook at 0x11444b0d0>, ‘output_file’: <function output_file at 0x11444b048>, ‘show’: <function show at 0x11444b598>, ‘save’: <function save at 0x11444b1e0>, ‘reset_output’: <function reset_output at 0x11444b158>}, ‘post_check’: <function CodeHandler.modify_document.<locals>.post_check at 0x1017d8840>}, {‘post_check’: <function CodeHandler.modify_document.<locals>.post_check at 0x1017d8840>, ‘module’: <module ‘bk_script_1001’ from ‘/Users/cbarber/ws/fabml/panel_apps/bug.py’>, ‘self’: <bokeh.application.handlers.code_runner.CodeRunner object at 0x10181ca58>, ‘_cwd’: ‘/Users/cbarber/ws/fabml/panel_apps’, ‘_sys_path’: [‘/Users/cbarber/ws/fabml/panel_apps’, ‘/Applications/PyCharm.app/Contents/helpers/pydev’, ‘/Users/cbarber/ws/fabml’, ‘/Users/cbarber/ws/fabml/src/test/python’, ‘/Users/cbarber/ws/fabml/src/main/python’, ‘/Applications/PyCharm.app/Contents/helpers/third_party/thriftpy’, ‘/Applications/PyCharm.app/Contents/helpers/pydev’, ‘/Users/cbarber/Library/Caches/PyCharm2019.1/cythonExtensions’, ‘/Users/cbarber/ws/fabml/panel_apps’, ‘/miniconda3/envs/fabml-dev/lib/python36.zip’, ‘/miniconda3/envs/fabml-dev/lib/python3.6’, ‘/miniconda3/envs/fabml-dev/lib/python3.6/lib-dynload’, ‘/miniconda3/envs/fabml-dev/lib/python3.6/site-packages’], ‘_sys_argv’: [‘/Users/cbarber/ws/fabml/panel_apps/bug.py’]}]

[analog-cbarber]

I am mostly arguing against closing the bug without fixing it. It is totally fine to have low-priority open bugs. Leaving them open is usually enough for developers to find the description and understand what is going on without debugging from scratch, but people are less likely to search through closed bug tickets for active problems.

I do not know that this bug is unique to the PyCharm debugger. That is just what I happened to be using. The extra reference in my case appeared to come from the frames f_locals attribute, which is a standard frame attribute (see https://docs.python.org/3/reference/datamodel.html#frame-objects), so I would expect the same thing could happen with any debugger. It will just depend on whether a given implementation uses the gc to manage the frame objects.

The behavior is reproducible in PyCharm using the script I provided, so it should be pretty easy to try other debuggers provided you have them at hand. I just tried using python -m pdb and did not see the problem (presumably it does not manage frame objects using the gc).

In any case, this should not be all that hard to fix, at least for this case, and I will give it a shot when I have a chance, but I have only just started using bokeh and have not gotten around to building it from source, so it may be a while before I get around to it.

So I recommend you just leave the bug as is and wait for me (or someone else) to come up with a fix and verify that it works.

[bryevdv]

I’m going to go ahead and close this at this point to clean up the tracker. It’s here as documentation if anyone else ever runs in to it (tho I have not heard of that happening). @analog-cbarber if you are ever able to submit a PR we would be pleased to accept it.