How to log the gradient and weights of the actor policy in TD3 using tensorboard?

Code snippet:

import tensorflow as tf
def Callback(_locals, _globals):
    self_ = _locals['self']
    tf.summary.histogram("Actor Network Weights Histogram", self_.policy_out.policy.??? )

I cannot understand from the

class FeedForwardPolicy(TD3Policy):

in FeedForwardPolicy(TD3Policy)

how to get the the trainable variables for the network, something like tf.trainable_variables() used to plot historgams in tensorboard.

[Miffyli]

@yotamitai

Use get_parameters(), which is right next to get_parameter_list().

[araffin]

Hello,

You should take a look at the code of TD3, not the policy, that’s where we use tf.trainable_variables(). (I recommend you to take a look at PPO2 too, where we log more things using tensorboard) You can get the name of the weights (scope + name) using model.get_parameter_list() (cf doc)

Since https://github.com/hill-a/stable-baselines/issues/409, we also added some documentation on how to log additional variables.