question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Starting tensorboard inline within a Jupyter notebook consistently times out

See original GitHub issue

Consider Stack Overflow for getting support using TensorBoard—they have a larger community with better searchability:

https://stackoverflow.com/questions/tagged/tensorboard

Do not use this template for for setup, installation, or configuration issues. Instead, use the “installation problem” issue template:

https://github.com/tensorflow/tensorboard/issues/new?template=installation_problem.md

To report a problem with TensorBoard itself, please fill out the remainder of this template.

Environment information (required)

Please run diagnose_tensorboard.py (link below) in the same environment from which you normally run TensorFlow/TensorBoard, and paste the output here:

Diagnostics

Diagnostics output
--- check: autoidentify
INFO: diagnose_tensorboard.py version 724b56cee52e7d8eb89bbeec1f0d5ce3e38c9682

--- check: general
INFO: sys.version_info: sys.version_info(major=3, minor=8, micro=6, releaselevel='final', serial=0)
INFO: os.name: nt
INFO: os.uname(): N/A
INFO: sys.getwindowsversion(): sys.getwindowsversion(major=10, minor=0, build=19042, platform=2, service_pack='')

--- check: package_management
INFO: has conda-meta: False
INFO: $VIRTUAL_ENV: None

--- check: installed_packages
INFO: installed: tensorboard==2.3.0
INFO: installed: tensorflow==2.3.1
INFO: installed: tensorflow-estimator==2.3.0

--- check: tensorboard_python_version
INFO: tensorboard.version.VERSION: '2.3.0'

--- check: tensorflow_python_version
2020-11-09 11:08:55.313364: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
2020-11-09 11:08:55.314312: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
INFO: tensorflow.__version__: '2.3.1'
INFO: tensorflow.__git_version__: 'v2.3.0-54-gfcc4b966f1'

--- check: tensorboard_binary_path
INFO: which tensorboard: b'C:\\Users\\huer\\AppData\\Local\\Programs\\Python\\Python38\\Scripts\\tensorboard.exe\r\n'

--- check: addrinfos
socket.has_ipv6 = True
socket.AF_UNSPEC = <AddressFamily.AF_UNSPEC: 0>
socket.SOCK_STREAM = <SocketKind.SOCK_STREAM: 1>
socket.AI_ADDRCONFIG = <AddressInfo.AI_ADDRCONFIG: 1024>
socket.AI_PASSIVE = <AddressInfo.AI_PASSIVE: 1>
Loopback flags: <AddressInfo.AI_ADDRCONFIG: 1024>
Loopback infos: [(<AddressFamily.AF_INET6: 23>, <SocketKind.SOCK_STREAM: 1>, 0, '', ('::1', 0, 0, 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 0, '', ('127.0.0.1', 0))]
Wildcard flags: <AddressInfo.AI_PASSIVE: 1>
Wildcard infos: [(<AddressFamily.AF_INET6: 23>, <SocketKind.SOCK_STREAM: 1>, 0, '', ('::', 0, 0, 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 0, '', ('0.0.0.0', 0))]

--- check: readable_fqdn
INFO: socket.getfqdn(): 'Surface-Laptop'

--- check: stat_tensorboardinfo
INFO: directory: C:\Users\huer\AppData\Local\Temp\.tensorboard-info
INFO: os.stat(...): os.stat_result(st_mode=16895, st_ino=3940649676184517, st_dev=443364023, st_nlink=1, st_uid=0, st_gid=0, st_size=8192, st_atime=1604948751, st_mtime=1604948698, st_ctime=1603511741)
INFO: mode: 0o40777

--- check: source_trees_without_genfiles
INFO: tensorboard_roots (1): ['C:\\Users\\huer\\AppData\\Local\\Programs\\Python\\Python38\\lib\\site-packages']; bad_roots (0): []

--- check: full_pip_freeze
INFO: pip freeze --all:
absl-py==0.11.0
appdirs==1.4.4
argon2-cffi==20.1.0
astunparse==1.6.3
async-generator==1.10
attrs==20.2.0
backcall==0.2.0
black==20.8b1
bleach==3.2.1
cachetools==4.1.1
certifi==2020.6.20
cffi==1.14.3
chardet==3.0.4
click==7.1.2
colorama==0.4.4
debugpy==1.1.0
decorator==4.4.2
defusedxml==0.6.0
entrypoints==0.3
Flask==1.1.2
gast==0.3.3
google-auth==1.23.0
google-auth-oauthlib==0.4.2
google-pasta==0.2.0
grpcio==1.33.2
h5py==2.10.0
idna==2.10
ipykernel==5.3.4
ipython==7.19.0
ipython-genutils==0.2.0
ipywidgets==7.5.1
itsdangerous==1.1.0
jedi==0.17.2
Jinja2==2.11.2
jsonschema==3.2.0
jupyter==1.0.0
jupyter-client==6.1.7
jupyter-console==6.2.0
jupyter-core==4.6.3
jupyterlab-pygments==0.1.2
Keras-Preprocessing==1.1.2
Markdown==3.3.3
MarkupSafe==1.1.1
mistune==0.8.4
mypy-extensions==0.4.3
nbclient==0.5.1
nbconvert==6.0.7
nbformat==5.0.8
nest-asyncio==1.4.2
notebook==6.1.4
numpy==1.18.5
oauthlib==3.1.0
opt-einsum==3.3.0
packaging==20.4
pandas==1.1.3
pandocfilters==1.4.3
parso==0.7.1
pathspec==0.8.0
pickleshare==0.7.5
pip==20.2.1
prometheus-client==0.8.0
prompt-toolkit==3.0.8
protobuf==3.13.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
Pygments==2.7.2
pyparsing==2.4.7
pyrsistent==0.17.3
python-dateutil==2.8.1
pytz==2020.4
pywin32==228
pywinpty==0.5.7
pyzmq==19.0.2
qtconsole==4.7.7
QtPy==1.9.0
regex==2020.9.27
requests==2.24.0
requests-oauthlib==1.3.0
rsa==4.6
selenium==3.141.0
Send2Trash==1.5.0
setuptools==49.2.1
six==1.15.0
tensorboard==2.3.0
tensorboard-plugin-wit==1.7.0
tensorflow==2.3.1
tensorflow-estimator==2.3.0
termcolor==1.1.0
terminado==0.9.1
testpath==0.4.4
toml==0.10.1
tornado==6.1
traitlets==5.0.5
typed-ast==1.4.1
typing-extensions==3.7.4.3
urllib3==1.25.11
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==1.0.1
wheel==0.35.1
widgetsnbextension==3.5.1
wrapt==1.12.1

For browser-related issues, please additionally specify:

  • Browser type and version (e.g., Chrome 64.0.3282.140):
  • Screenshot, if it’s a visual issue:

Issue description

Please describe the bug as clearly as possible. How can we reproduce the problem without additional resources (including external data files and proprietary Python modules)?

Repro steps:

  1. Install Python 3.8.6 64-bit on Windows 10
  2. Run python -m pip install jupyter tensorflow
  3. Run jupyter notebook
  4. Create a Jupyter notebook in the browser
  5. Run %load_ext tensorboard in one cell, then %tensorboard --logdir logs/fit in a second cell
  6. I’d expect to see the tensorboard website appear inline with the message about no active dashboards. Instead, I get a message about timing out waiting for TensorBoard to start:

image 7. If I rerun the cell with %tensorboard --logdir logs/fit, tensorboard does show up inline.

The diagnostic info above is based on my local machine. I initially thought my local box might simply have a busted tensorboard install, but the problem persisted even after uninstalling and reinstalling tensorflow and tensorboard. I was able to repro this on a clean Windows VM with the above repro steps.

Debugging the iopub messages sent by the Jupyter kernel, the Jupyter kernel does send a message with the ‘Launching TensorBoard…’ message in response to the execution request for %tensorboard --logdir logs/fit the first time, but it doesn’t ever send the iframe back. For some reason requesting the tensorboard launch again results in an immediate response about ‘Reusing TensorBoard on port <…>’, followed by a message with the iframe for display in the cell output.

Happy to provide additional information that will help with diagnosing the issue!

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:8 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
wchargincommented, Dec 1, 2020

Hi @joyceerhl—we think we have a fix for this in #4407. It should go out in today’s tb-nightly, or you can build your own Pip package from head with bazel build //tensorboard/pip_package, or you can manually patch the diff into your virtualenv (it’s a small change to one Python file). If this doesn’t fix it on your end, please let us know and we can reopen this. Thanks!

0reactions
wchargincommented, Dec 1, 2020

Excellent! Thanks for letting us know.

Read more comments on GitHub >

github_iconTop Results From Across the Web

tensor board output is not shown when I am trying to run the ...
Do note that there is a known problem with TensorBoard on Windows where launching TensorBoard inline in a Jupyter notebook will time out...
Read more >
Using TensorBoard in Notebooks - TensorFlow
Start by installing TF 2.0 and loading the TensorBoard notebook extension: For Jupyter users: If you've installed Jupyter and TensorBoard ...
Read more >
a new way to use TensorBoard on Databricks
The %tensorboard command starts a TensorBoard server and embeds the TensorBoard user interface inside the Databricks notebook for data ...
Read more >
Trying To Run Tensorboard For The First Time - ADocLib
Start by installing TF 2.0 and loading the TensorBoard notebook extension: For Jupyter users: If you've installed Jupyter and TensorBoard into. Inline ......
Read more >
Top JupyterLab Extensions for Machine Learning Research
A Jupyter notebook add-on called JupyterLab system monitor shows system ... tensorboard to start, manage, and stop in the jupyter interface, ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found