[docs] Issue on `rllib.rst`

Documentation Problem/Question/Comment

I’m getting started on the RLlib tutorial. I attempted to run a python script:

from ray import tune
from ray.rllib.agents.ppo import PPOTrainer
tune.run(PPOTrainer, config={"env": "CartPole-v0"})  # "eager": True for eager execution

And I got the following error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_summary_ops.py", line 851, in write_summary
    tensor, tag, summary_metadata)
tensorflow.python.eager.core._FallbackException: This function does not handle the case of the path where all inputs are not already EagerTensors.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/ray/tune/trial_runner.py", line 540, in _process_trial
    result, terminate=(decision == TrialScheduler.STOP))
  File "/usr/local/lib/python3.7/site-packages/ray/tune/trial.py", line 386, in update_last_result
    self.result_logger.on_result(self.last_result)
  File "/usr/local/lib/python3.7/site-packages/ray/tune/logger.py", line 333, in on_result
    _logger.on_result(result)
  File "/usr/local/lib/python3.7/site-packages/ray/tune/logger.py", line 193, in on_result
    "/".join(path + [attr]), value, step=step)
  File "/usr/local/lib/python3.7/site-packages/tensorboard/plugins/scalar/summary_v2.py", line 65, in scalar
    metadata=summary_metadata)
  File "/usr/local/lib/python3.7/site-packages/tensorflow_core/python/ops/summary_ops_v2.py", line 646, in write
    _should_record_summaries_v2(), record, _nothing, name="summary_cond")
  File "/usr/local/lib/python3.7/site-packages/tensorflow_core/python/framework/smart_cond.py", line 54, in smart_cond
    return true_fn()
  File "/usr/local/lib/python3.7/site-packages/tensorflow_core/python/ops/summary_ops_v2.py", line 640, in record
    name=scope)
  File "/usr/local/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_summary_ops.py", line 856, in write_summary
    writer, step, tensor, tag, summary_metadata, name=name, ctx=_ctx)
  File "/usr/local/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_summary_ops.py", line 893, in write_summary_eager_fallback
    attrs=_attrs, ctx=_ctx, name=name)
  File "/usr/local/lib/python3.7/site-packages/tensorflow_core/python/eager/execute.py", line 76, in quick_execute
    raise e
  File "/usr/local/lib/python3.7/site-packages/tensorflow_core/python/eager/execute.py", line 61, in quick_execute
    num_outputs)
TypeError: An op outside of the function building code is being passed
a "Graph" tensor. It is possible to have Graph tensors
leak out of the function building context by including a
tf.init_scope in your function building code.
For example, the following function will fail:
  @tf.function
  def has_init_scope():
    my_constant = tf.constant(1.)
    with tf.init_scope():
      added = my_constant * 2
The graph tensor has name: create_file_writer/SummaryWriter:0

Not sure if this is a bad environment setup on my part or if the tutorial is outdated.

(Created directly from the docs)

[RedTachyon]

I’m getting the same error with TF 2, but not with TF 1.14. I’m using ray 0.7.5

[RedTachyon]

Like in #5934 upgrading to 0.8.0.dev6 fixes this issue.