[BUG] Network latency results in orphaned embedded divs for Bokeh Server

Before I get into the bug report, I would like to thank the core Bokeh team for their hard work on this project - increasingly it’s my Python plotting tool of choice!

Software Version Info (bokeh, python, notebook, OS, browser, any other relevant packages)

• Bootstrap 4.0.0
• Bokeh: 1.4.0
• Python: 3.6.8
• OS: Ubuntu Linux 19.10
• Browser: Firefox 70.0.1, Chrome 78.0.3904.70

Description of expected behavior and the observed behavior

Expected Behaviour

Provided the network is fairly snappy, everything works as it should, and my Bokeh server plots are properly located within my HTML template, e.g.

<div class="bk-root" id="a080f844-164a-495b-b7c3-848fd2c9342c" data-root-id="1432">
  <div class="bk" style="position: relative; display: block; left: 0px; top: 0px; width: 500px; height: 250px; margin: 0px;">
    <div class="bk" style="width: 618px; height: 309px; cursor: default;">
      <!-- rest of plot's divs here -->
    </div>
  </div>
</div>

Observed Behaviour

However, when there is latency on the network, or the Bokeh server is under strain (I’ve reproduced this by DoSing with many concurrent browser tabs), the bokeh plots are not properly placed in the designated place in the Jinja template, but rather appended to the end of the document’s body. So the plots are rendering, just in the wrong place, e.g.

<div class="bk-root" id="f98d93cb-905f-4d52-8fbf-9e7f0393b631" data-root-id="1373"></div>
<!-- Rest of my HTML body -->
<div class="bk bk-root">
    <div class="bk" style="position: relative; display: block; left: 0px; top: 0px; width: 948px; height: 474px; margin: 0px;">
        <div class="bk" style="width: 1354px; height: 677px;">
          <!-- rest of plot's divs here -->
        </div>
    </div>
</div>

As I’m doing all of my styling and layout in a HTML template using Bootstrap, this results in a pretty broken WUI.

Complete, minimal, self-contained example code that reproduces the issue

I’ve struggled to reproduce this with a minimalist example, as I suspect is being caused by the size of the rendered plots, etc.

I can share all of my code, however I’m concerned that parsing that would slow down this request - I have a strong suspicion about where the problem is here. See the theory subsection for more detail.

Browser JavaScript console output

From unhappy case:

[bokeh] setting log level to: 'info' logging.js:1
[bokeh] Websocket connection 0 is now open connection.js:1
Source map error: Error: request failed with status 404
Resource URL: <network path to app>/static/css/bootstrap.min.css
Source Map URL: bootstrap.min.css.map
Use of mozImageSmoothingEnabled is deprecated. Please use the unprefixed imageSmoothingEnabled property instead. canvas.js:1
[bokeh] document idle at 382 ms document.js:1
Bokeh items were rendered successfully index.js:1

I don’t think the missing bootstrap file is relevant, but happy to be corrected on this.

Is the line document idle at 382 ms relevant?

Screenshots of the bug in action

Expected

Observed

The empty div is present, but compressed into a single line. You can actually see one of the orphaned divs in the background, out of place.

Theory

caveat I’m not much of a web-dev, so I might be horribly off base here.

As mentioned earlier, I have a suspicion the problem is here. Tracing the code through, it looks like the only place where the bokeh item either ends up in the right place, or just appended to document.body.

My theory is that there is some sort of race condition, where the client side Bokeh JS is trying to embed the plot, and somehow the document.getElementById here is not returning the right div, and hence it’s defaulting to document.body.

I suppose another possibility is that Bootstrap is somehow interfering with the Bokeh JS finding the right div?

I’m open to working on a patch for this, but unfortunately I don’t know TypeScript, so if someone is willing to provide a fair bit of guidance, I could give it a bash?

[Gordonei]

OK, looks like the fix solves this issue!

Baseline

1. Cloned the repo anew
2. Checked out b19f2c5547024bdc288d02e73fdb65e65991df5f (1.4.0)
3. Did a fresh install (i.e. selected choice (1) for build and install fresh)
4. Checked version (python3 -m bokeh --version): 1.4.0
5. Started up a server instance of my app with two processes: BOKEH_LOG_LEVEL='debug' python3 -m bokeh serve city_dashboard_app --num-procs=2 --log-level debug
6. DoSed the server using firefox: for i in 1 2 3 4 5 6 7 8 9 10; do firefox -new-tab -url http://localhost:5006; done

Confirmed that the plots got misplaced.

With #9660

1. Checked out e0b465a12162170087bc5adc08689fd3ff602e3c (the commit with the fix in it)
2. Did a fresh install (i.e. selected choice (1) for build and install fresh)
3. Checked version (python3 -m bokeh --version): 2.0.0dev6-41-ge0b465a12-dirty
4. Started up a server instance of my app with two processes: BOKEH_LOG_LEVEL='debug' python3 -m bokeh serve city_dashboard_app --num-procs=2 --log-level debug
5. DoSed the server using firefox: for i in 1 2 3 4 5 6 7 8 9 10; do firefox -new-tab -url http://localhost:5006; done

No sign of misplaced divs!

Thanks to @philippjfr and @mattpap for sorting this out!

[Gordonei]

Thanks guys, reading through the above PR, it does seem to be hitting the issue I was describing.

I’ll try with my code against master - failing that, I’ll try provide a simple reproduction.