TD3 IndentationError/OperatorNotAllowedInGraphError

Hi, I’ve been working with SAC up to now and it works fine with my custom environment. However, I wanted to test some other algorithms but I run into the following error that I have not really any idea of where to begin to check. The algorithm runs properly for as long as it has “warmup” steps, as soon as it ends, it crashes with the following message.

WARNING:tensorflow:AutoGraph could not transform <bound method TD3._train_body of <tensorflow.python.eager.function.TfMethodTarget object at 0x7ff6304e2b00>> and will run it as-is. Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, export AUTOGRAPH_VERBOSITY=10) and attach the full output. Cause: unexpected indent (<unknown>, line 16) Traceback (most recent call last): File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/impl/api.py”, line 526, in converted_call converted_f = conversion.convert(target_entity, program_ctx) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/impl/conversion.py”, line 326, in convert free_nonglobal_var_names) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/impl/conversion.py”, line 240, in _convert_with_cache entity, program_ctx) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/impl/conversion.py”, line 475, in convert_entity_to_ast nodes, name, entity_info = convert_func_to_ast(o, program_ctx) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/impl/conversion.py”, line 634, in convert_func_to_ast node, source = parser.parse_entity(f, future_features=future_features) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/pyct/parser.py”, line 207, in parse_entity return _attempt_to_parse_normal_source(source, future_features) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/pyct/parser.py”, line 111, in _attempt_to_parse_normal_source return parse_str(source, preamble_len=len(future_features)), source File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/pyct/parser.py”, line 224, in parse_str module_node = gast.parse(src) File “/usr/local/lib/python3.6/dist-packages/gast/gast.py”, line 240, in parse return ast_to_gast(_ast.parse(*args, **kwargs)) File “/usr/lib/python3.6/ast.py”, line 35, in parse return compile(source, filename, mode, PyCF_ONLY_AST) File “<unknown>”, line 16 with tf.GradientTape() as tape: ^ IndentationError: unexpected indent During handling of the above exception, another exception occurred: Traceback (most recent call last): File “/root/ros_ws/src/ros-universal-robots/ur3e_rl/scripts/tf2rl_td3.py”, line 130, in <module> trainer() File “/root/pylibs/tf2rl/tf2rl/experiments/trainer.py”, line 122, in call None if not self._use_prioritized_rb else samples[“weights”]) File “/root/pylibs/tf2rl/tf2rl/algos/ddpg.py”, line 119, in train states, actions, next_states, rewards, done, weights) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/def_function.py”, line 568, in call result = self._call(*args, **kwds) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/def_function.py”, line 615, in _call self._initialize(args, kwds, add_initializers_to=initializers) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/def_function.py”, line 497, in _initialize *args, **kwds)) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/function.py”, line 2389, in _get_concrete_function_internal_garbage_collected graph_function, _, _ = self._maybe_define_function(args, kwargs) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/function.py”, line 2703, in _maybe_define_function graph_function = self._create_graph_function(args, kwargs) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/function.py”, line 2593, in _create_graph_function capture_by_value=self._capture_by_value), File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/func_graph.py”, line 978, in func_graph_from_py_func func_outputs = python_func(*func_args, **func_kwargs) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/def_function.py”, line 439, in wrapped_fn return weak_wrapped_fn().wrapped(*args, **kwds) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/function.py”, line 3211, in bound_method_wrapper return wrapped_fn(*args, **kwargs) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/func_graph.py”, line 964, in wrapper user_requested=True, File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/impl/api.py”, line 560, in converted_call return call_unconverted(f, args, kwargs, options) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/impl/api.py”, line 326, in call_unconverted return f.self.call(args, kwargs) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/function.py”, line 3173, in call return wrapped_fn(self.weakrefself_target(), *args, **kwargs) File “/root/pylibs/tf2rl/tf2rl/algos/td3.py”, line 94, in _train_body if tf.math.equal(self._it % self._actor_update_freq, 0): File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py”, line 757, in bool self._disallow_bool_casting() File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py”, line 523, in _disallow_bool_casting “using a tf.Tensor as a Python bool”) File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py”, line 510, in _disallow_when_autograph_enabled " decorating it directly with @tf.function.".format(task)) tensorflow.python.framework.errors_impl.OperatorNotAllowedInGraphError: using a tf.Tensor as a Python bool is not allowed: AutoGraph did not convert this function. Try decorating it directly with @tf.function.

Have you experienced this error before?

[keiohta]

@cambel I fixed the bug on issue83 branch, so now you can run the code without the error. Please try first:

$ git checkout issue83
# Please reinstall tf2rl if you have not installed with developer mode
$ pip install -e .
$ python examples/run_td3.py

However, the issue83 branch has not passed all unittests, so I cannot ensure the version works properly (actually it includes some bugs around inverse RL, such as GAIL, but I think other part should be fine).

I’ll close this issue if you successfully run your code, and then keep fixing bugs around tf2.1 on issue 84.

[keiohta]

@cambel Thanks for closing the issue! FYI: I merged the issue83 branch to master, so you can also install from pypi if you want:

$ pip install -U tf2rl