TensorFlow keras callback using tensorboard, "ProfilerNotRunningError: Cannot stop profiling. No profiler is running."

Environment information (required)

System: Windows 10, RAM 64 GB Environment: Python 3.6, TensorFlow-gpu==2.0.0-alpha0 GPU: GeForce RTX 2080 Ti, CUDA 10.0, cudnn-10.0-windows10-x64-v7.4.2.24

Issue description

When I try the Guide in https://tensorflow.google.cn/tensorboard/r2/graphs, a mistake is often encountered, such as a simple example:

@tf.function
def my_func(x, y):
    return tf.nn.relu(tf.matmul(x, y))

stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
logdir = os.path.join(PATH,'func',stamp)
writer = tf.summary.create_file_writer(logdir)

x = tf.random.uniform((3, 3))
y = tf.random.uniform((3, 3))

tf.summary.trace_on(graph=True, profiler=True)

z = my_func(x, y)

with writer.as_default():
    tf.summary.trace_export(
        name="my_func_trace",
        step=0,
        profiler_outdir=logdir)

Error description

ProfilerNotRunningError                   Traceback (most recent call last)
<ipython-input-11-c67128fece27> in <module>
     19         name="my_func_trace",
     20         step=0,
---> 21         profiler_outdir=logdir)
     22 

c:\users\admin\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\ops\summary_ops_v2.py in trace_export(name, step, profiler_outdir)
   1142 
   1143   if profiler:
-> 1144     _profiler.save(profiler_outdir, _profiler.stop())
   1145 
   1146   trace_off()

c:\users\admin\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\eager\profiler.py in stop()
     97     if _profiler is None:
     98       raise ProfilerNotRunningError(
---> 99           'Cannot stop profiling. No profiler is running.')
    100     with c_api_util.tf_buffer() as buffer_:
    101       pywrap_tensorflow.TFE_ProfilerSerializeToString(

ProfilerNotRunningError: Cannot stop profiling. No profiler is running.

I don’t know what is issue mean, and how to fix it.

[jdc17d]

I was having the same issue. I am using anaconda and in jupyter notebook when I tried to set up the tensorboard callback I was getting the same error. My import tensorboard command was running without complaint but I don’t think it was really there.

I did conda install tensorboard and I’ve been able to run without issues. I also just specify my dir as ‘logs’ so I don’t have to thing about the forward/backslash as I am on a windows machine.

If you aren’t using conda, maybe just try pip install tensorboard and see if that fixes your issue!

[wchargin]

That sounds like you’re running into tensorflow/tensorflow#26021, then: https://github.com/tensorflow/tensorflow/issues/26021

TL;DR: Due to a current bug in TensorFlow, on Windows, if your path contains any forward slashes then it should not contain any backslashes. Thus, os.path.join(logdir, "foo") will hit this bug if logdir contains any forward slashes.

In the meantime until this bug is fixed, please ensure that your paths use consistent delimiters.