Delay between autoload.js and websocket request

READ AND FOLLOW THESE INSTRUCTIONS CAREFULLY

ISSUES THAT DO NOT CONTAIN NECESSARY INFORMATION MAY BE CLOSED, IMMEDIATELY

The issue tracker is NOT the place for general support. For questions and technical assistance, come ask the Bokeh mailing list or join the chat on Gitter. For feature requests, please provide a detailed description or proposal of the new capability or behavior.

For defects or deficiencies, please provide ALL OF THE FOLLOWING:

ALL software version info (bokeh, python, notebook, OS, browser, any other relevant packages)

bokeh 1.0.2 python 3.6.5 OS CentOS-7.5.1804

Description of expected behavior and the observed behavior

For whatever reason, it appears that on some requests, there can be a significant delay between the autoload.js request and the subsequent websocket connection. Normally, this process takes no more than 1-2 seconds:

doc.session_context.request.arguments: {'bokeh-autoload-element': [b'1088'], 'bokeh-app-path': [b'/graphs/enviz_graphs'], 'bokeh-absolute-url': [b'https://*redacted*/graphs/enviz_graphs'], 'processor_id': [b'83,187,196,1114,206,335,536,212,214,1173,217,250,252,256,265,876,268,298,999']}
2019-01-18 22:44:45,794 root_url should end with a /, adding one
2019-01-18 22:44:45,797 200 GET /graphs/enviz_graphs/autoload.js?bokeh-autoload-element=1089&bokeh-app-path=/graphs/enviz_graphs&bokeh-absolute-url=https://*redacted*/graphs/enviz_graphs&processor_id=83%2C187%2C196%2C1114%2C206%2C335%2C536%2C212%2C214%2C1173%2C217%2C250%2C252%2C256%2C265%2C876%2C268%2C298%2C999 (10.50.1.159) 398.52ms
2019-01-18 22:44:47,291 101 GET /graphs/enviz_graphs/ws?bokeh-protocol-version=1.0&bokeh-session-id=ImqIQZ1sbiZS4KsAOocVHGFgUGfJJLwHxG44Irv9Xls9&pid=83,187,196,1114,206,335,536,212,214,1173,217,250,252,256,265,876,268,298,999 (10.50.1.159) 0.56ms
2019-01-18 22:44:47,291 WebSocket connection opened
2019-01-18 22:44:47,291 Receiver created for Protocol('1.0')
2019-01-18 22:44:47,291 ProtocolHandler created for Protocol('1.0')
2019-01-18 22:44:47,291 ServerConnection created
2019-01-18 22:44:47,350 Sending pull-doc-reply from session 'ImqIQZ1sbiZS4KsAOocVHGFgUGfJJLwHxG44Irv9Xls9'

Notice the autoload request at 22:44:45 and the ws request at 22:44:47. (2 seconds)

However, sometimes the ws request can arrive nearly a minute later:

doc.session_context.request.arguments: {'bokeh-autoload-element': [b'1090'], 'bokeh-app-path': [b'/graphs/enviz_graphs'], 'bokeh-absolute-url': [b'https://*redacted*/graphs/enviz_graphs'], 'processor_id': [b'83,187,196,1114,206,335,536,212,214,1173,217,250,252,256,265,876,268,298,300,1347,1350,1352,284,307,1115,1229,999,92,']}
2019-01-18 22:45:10,741 root_url should end with a /, adding one
2019-01-18 22:45:10,745 200 GET /graphs/enviz_graphs/autoload.js?bokeh-autoload-element=1090&bokeh-app-path=/graphs/enviz_graphs&bokeh-absolute-url=https://*redacted*/graphs/enviz_graphs&processor_id=83%2C187%2C196%2C1114%2C206%2C335%2C536%2C212%2C214%2C1173%2C217%2C250%2C252%2C256%2C265%2C876%2C268%2C298%2C300%2C1347%2C1350%2C1352%2C284%2C307%2C1115%2C1229%2C999%2C92%2C (10.50.1.159) 392.75ms
2019-01-18 22:45:35,357 Scheduling 1 sessions to discard
2019-01-18 22:45:35,357 Discarding session '1fz6E0KuyuaCaCscdyKLyI2YJze38csKNckNQkotkrE8' last in use 24616.089113235474 milliseconds ago
2019-01-18 22:45:35,358 Deleting 1 modules for <bokeh.document.document.Document object at 0x7f8bb89f8438>
2019-01-18 22:45:50,352 [pid 11775] 1 clients connected
2019-01-18 22:45:50,352 [pid 11775]   /enviz_graphs has 1 sessions with 0 unused
2019-01-18 22:46:05,562 101 GET /graphs/enviz_graphs/ws?bokeh-protocol-version=1.0&bokeh-session-id=1fz6E0KuyuaCaCscdyKLyI2YJze38csKNckNQkotkrE8&pid=83,187,196,1114,206,335,536,212,214,1173,217,250,252,256,265,876,268,298,300,1347,1350,1352,284,307,1115,1229,999,92, (10.50.1.159) 0.58ms
2019-01-18 22:46:05,562 WebSocket connection opened
doc.session_context.request.arguments: {'pid': [b'83,187,196,1114,206,335,536,212,214,1173,217,250,252,256,265,876,268,298,300,1347,1350,1352,284,307,1115,1229,999,92,']}
2019-01-18 22:46:05,563 Error running application handler <bokeh.application.handlers.directory.DirectoryHandler object at 0x7f8bb8f75cf8>: local variable 'current_pids' referenced before assignment
File "env_frontend.py", line 30, in modify_doc:
if len(current_pids) < 1: Traceback (most recent call last):
File "/*redacted*/enviz/venv/lib64/python3.6/site-packages/bokeh/application/handlers/code_runner.py", line 180, in run
exec(self._code, module.__dict__)
File "/*redacted*/enviz/venv/lib/python3.6/site-packages/enviz_graphs/main.py", line 7, in <module>
modify_doc(doc)
File "/*redacted*/enviz/venv/lib/python3.6/site-packages/enviz_graphs/env_frontend.py", line 30, in modify_doc
if len(current_pids) < 1:
UnboundLocalError: local variable 'current_pids' referenced before assignment
2019-01-18 22:46:05,563 Receiver created for Protocol('1.0')
2019-01-18 22:46:05,563 ProtocolHandler created for Protocol('1.0')
2019-01-18 22:46:05,563 ServerConnection created
2019-01-18 22:46:05,631 Sending pull-doc-reply from session '1fz6E0KuyuaCaCscdyKLyI2YJze38csKNckNQkotkrE8'

Notice the autoload request at 22:45:10 and the ws request at 22:46:05. (55 seconds)

In that gap, it appears that the session created by the autoload request was discarded as being unused at 22:46:05. (we have the default 15000 ms timeout for that.)

In both cases, the request for autoload.js takes less than 400 ms, so the slowdown seems like it would be in the browser, though I don’t yet have any browser profiling that caught it.

Then, when the ws request comes in, it tries to create a new session, but fails to run our module, as the correct keys aren’t in doc.session_context.request.arguments.

After this, every request to the bokeh server fails at requesting autoload until we restart the server, as it appears that doc.session_context.request.arguments is always None after that.

Complete, minimal, self-contained example code that reproduces the issue

N/A

Stack traceback and/or browser JavaScript console output

N/A

Screenshots or screencasts of the bug in action

N/A

[gotyaoi]

Yeah, switching self.failed to self._lifecycle_handler.failed also seems to work. I will submit a PR in a little bit.

[bryevdv]

@gotyaoi I think the issue is that the failed property encompasses too much (both script and lifecycle handler failure). I think that at the top of modify_doc there, we do want to bail if self._lifecycle_handler has failed (but should not check the script handler since it has not even been run this time around yet). Can you test out that change, and if it works, a PR would be great.