Cannot launch TensorBoard from source due to debugger plugin

TensorBoard master, with TensorFlow 1.3.0 from pip, cannot run: it fails to import a Python library related to gRPC.

The error is:

Traceback (most recent call last):
  File "/home/wchargin/.cache/bazel/_bazel_wchargin/3f99396cfb979f2f5a2059c1fd233f92/execroot/org_tensorflow_tensorboard/bazel-out/local-fastbuild/bin/tensorboard/tensorboard.runfiles/org_tensorflow_tensorboard/tensorboard/main.py", line 38, in <module>
    from tensorboard.plugins.debugger import debugger_plugin as debugger_plugin_lib
  File "/home/wchargin/.cache/bazel/_bazel_wchargin/3f99396cfb979f2f5a2059c1fd233f92/execroot/org_tensorflow_tensorboard/bazel-out/local-fastbuild/bin/tensorboard/tensorboard.runfiles/org_tensorflow_tensorboard/tensorboard/plugins/debugger/debugger_plugin.py", line 35, in <module>
    from tensorboard.plugins.debugger import debugger_server_lib
  File "/home/wchargin/.cache/bazel/_bazel_wchargin/3f99396cfb979f2f5a2059c1fd233f92/execroot/org_tensorflow_tensorboard/bazel-out/local-fastbuild/bin/tensorboard/tensorboard.runfiles/org_tensorflow_tensorboard/tensorboard/plugins/debugger/debugger_server_lib.py", line 33, in <module>
    from tensorflow.python.debug.lib import grpc_debug_server
ImportError: cannot import name grpc_debug_server

The first bad commit is (unsurprisingly) a856e61d39d231e38d45e32e92b28be596afbb58, which I identified by using git bisect with the following script:

#!/bin/bash
! bazel run tensorboard 2>&1 | grep -F 'cannot import name grpc_debug_server'

Steps to reproduce:

$ virtualenv /tmp/tensorflow-1.3.0-fresh
$ source /tmp/tensorflow-1.3.0-fresh/bin/activate
$ pip install tensorflow==1.3.0
$ git checkout b1a4d2586a0eae1ce7f3a18b4db188b62c4daaee  # current origin/master
$ bazel run tensorboard -- --logdir /tmp/data

The following patch fixes the problem:

diff --git a/tensorboard/main.py b/tensorboard/main.py
index ec84e25..fb5d2cd 100644
--- a/tensorboard/main.py
+++ b/tensorboard/main.py
@@ -35,7 +35,7 @@ from tensorboard.backend import application
 from tensorboard.backend.event_processing import event_file_inspector as efi
 from tensorboard.plugins.audio import audio_plugin
 from tensorboard.plugins.core import core_plugin
-from tensorboard.plugins.debugger import debugger_plugin as debugger_plugin_lib
+#from tensorboard.plugins.debugger import debugger_plugin as debugger_plugin_lib
 from tensorboard.plugins.distribution import distributions_plugin
 from tensorboard.plugins.graph import graphs_plugin
 from tensorboard.plugins.histogram import histograms_plugin
@@ -240,11 +240,12 @@ def main(unused_argv=None):
     efi.inspect(FLAGS.logdir, event_file, FLAGS.tag)
     return 0
   else:
-    def ConstructDebuggerPluginWithGrpcPort(context):
-      debugger_plugin = debugger_plugin_lib.DebuggerPlugin(context)
-      if FLAGS.debugger_data_server_grpc_port is not None:
-        debugger_plugin.listen(FLAGS.debugger_data_server_grpc_port)
-      return debugger_plugin
+    pass
+    #def ConstructDebuggerPluginWithGrpcPort(context):
+    #  debugger_plugin = debugger_plugin_lib.DebuggerPlugin(context)
+    #  if FLAGS.debugger_data_server_grpc_port is not None:
+    #    debugger_plugin.listen(FLAGS.debugger_data_server_grpc_port)
+    #  return debugger_plugin
 
     plugins = [
         core_plugin.CorePlugin,
@@ -258,7 +259,7 @@ def main(unused_argv=None):
         projector_plugin.ProjectorPlugin,
         text_plugin.TextPlugin,
         profile_plugin.ProfilePlugin,
-        ConstructDebuggerPluginWithGrpcPort,
+        #ConstructDebuggerPluginWithGrpcPort,
     ]
 
     tb = create_tb_app(plugins)

Versions:

$ bazel version
Build label: 0.5.4
Build target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Fri Aug 25 10:00:00 2017 (1503655200)
Build timestamp: 1503655200
Build timestamp as int: 1503655200
$ pip --version
pip 9.0.1 from /tmp/tensorflow-1.3.0-fresh/local/lib/python2.7/site-packages (python 2.7)
$ lsb_release -a
No LSB modules are available.
Distributor ID:	LinuxMint
Description:	Linux Mint 18.2 Sonya
Release:	18.2
Codename:	sonya

[jart]

Upgrading grpc and protobuf doesn’t fix the issue either. How stable is grpc? I’m concerned that issues like these could cause problems for TensorBoard and TensorFlow users if we make it a dependency. Should we rework the debugger code so that it can survive if importing grpc fails? Then have an “inactive plugin” page that tells the user to pip install grpc if he/she wants to use it?

[caisq]

I have some rough ideas of what might be the cause and how to fix it from the tensorflow side. Will give it a shot tomorrow.