Missing some output widgets when interact() calls a function which starts a thread which calls a function

(Opening issue as requested by Jason Grout in the Gitter channel)

My code calls ipywidgets.interact(), which calls an alpha() function. alpha() defines two inner functions: beta() and gamma(). alpha() displays a progress bar and starts gamma() in a new thread; gamma() waits for a timer to elapse and calls beta().

If beta() is the only function to render ipywidgets or call display(), then everything will seem to work correctly.

The problem: if alpha() or gamma() try to render ipywidgets or call display(), then the display() calls in beta() don’t always happen. Sometimes none of them add any outputs to the notebook, sometimes only some of the outputs are added, and in rare cases all of the outputs happen just fine. I have not seen any of the display() calls come in out of order, however.

My code:

import datetime
import threading
import time

from IPython.display import (
    HTML,
    display,
)
import ipywidgets

def alpha(whatever):

    def beta(something):
        display(HTML("<p>One {} {}</p>".format(something, datetime.datetime.now())))
        display(HTML("<p>Two {} {}</p>".format(something, datetime.datetime.now())))
        display(HTML("<p>Three {} {}</p>".format(something, datetime.datetime.now())))
        display(HTML("<p>Four {} {}</p>".format(something, datetime.datetime.now())))
        display(HTML("<p>Five {} {}</p>".format(something, datetime.datetime.now())))
        display(HTML("<p>Six {} {}</p>".format(something, datetime.datetime.now())))
        display(HTML("<p>Seven {} {}</p>".format(something, datetime.datetime.now())))
        display(HTML("<p>Eight {} {}</p>".format(something, datetime.datetime.now())))
        display(HTML("<p>Nine {} {}</p>".format(something, datetime.datetime.now())))
        display(HTML("<p>Ten {} {}</p>".format(something, datetime.datetime.now())))

    def gamma(a_thing, timerlength=3.0, updateinterval=0.2):
        ctr = 0
        #display(HTML('<p>Trying to break it from gamma()</p>'))
        while ctr < timerlength:
            time.sleep(updateinterval)
            ctr += updateinterval
        beta(a_thing)
    
    streetmap_lock = threading.RLock()

    #display(HTML('<p>Trying to break it from alpha()</p>'))

    mapthread = threading.Thread(
        target=gamma,
        args=(whatever,))
    mapthread.start()

ipywidgets.interact(
    alpha,
    whatever=ipywidgets.Text(
        value="WHATEVER",
        description="Whatever"))

When I copy and paste that into a Python3 cell as is, I can run the cell and wait three seconds and the ten display() calls from beta() come through every time. But when I uncomment the display() call in alpha() or gamma(), the ten calls in beta() are very inconsistent.

(In my real code, I want to display an IntProgress in alpha() or gamma() - it doesn’t matter which - and that’s when I tripped over this problem. Also in my real code is a global object that I am using to cancel the operation if the user changes a parameter before the timer is up, and restart it with the new parameter value.)

Note that the timer is 3 seconds long, which is certainly long enough that nothing is trying to call display() at the same time. To be absolutely sure, I tested it with a threading.RLock instance that was passed between the functions, and still found the same behavior described above.

Tested with:

• Python 3.6
• Jupyter 4.3.0
• Jupyter Notebook 5.2.0
• ipywidgets 7.0.3
• widgetsnbextension 3.0.6

I built my test case on the latest version of macOS, but I was seeing similar errors in my real (larger, messier) notebook under Windows and Linux as well.

Is this a bug? Or am I doing something wrong?

Thanks for any help

[pbugnion]

Great! Glad you got it working.

Adding this as an additional example at the end of Asynchronous widgets seems like a good idea (unless someone with more experience of the documentation can think of a more appropriate place). PRs definitely welcome!

[jasongrout]

Fixed by #1794.