Custom FeedForwardPolicy does not care for net_arch parameters

Describe the bug I am trying to create my own Custom Policy network, however when I check the tensorboard logs and look under graphs I can see that it does not create the network as specified after my cnn extraction: net_arch=[64, dict(pi=[64, 64], vf=[64, 64])],

Code example

class CustomPolicy(FeedForwardPolicy):

    def __init__(self, *args, **kwargs,):
        super(CnnExtractor, self).__init__(*args, **kwargs,
                                           cnn_extractor=custom_cnn,
                                           act_fun=tf.nn.relu,
                                           net_arch=[64, dict(pi=[64, 64],
                                                          vf=[64, 64])],
                                           feature_extraction='cnn', )


def custom_cnn(scaled_images, **kwargs):
    """

    :param scaled_images: (TensorFlow Tensor) Image input placeholder
    :param kwargs: (dict) Extra keywords parameters for the convolutional layers of the CNN
    :return: (TensorFlow Tensor) The CNN output layer
    """
    activ = tf.nn.relu
    layer_1 = activ(conv(scaled_images, 'c1', n_filters=18, filter_size=2, stride=1, **kwargs))
    layer_2 = activ(conv(layer_1, 'c2', n_filters=64, filter_size=2, stride=1, **kwargs))
    layer_3 = activ(conv(layer_2, 'c3', n_filters=64, filter_size=2, stride=1, **kwargs))
    layer_3 = conv_to_fc(layer_3)
    return activ(linear(layer_3, 'fc1', n_hidden=64))

I have attached the tensboard files here: logs.zip

System Info Describe the characteristic of your environment:

• Describe how the library was installed (pip, docker, source, …)
• • PIP
• GPU models and configuration
• • GTX 1070
• Python version
• • 3.7
• Tensorflow version
• 1.15.0

Additional context Add any other context about the problem here.

[araffin]

Wait so I can’t have my network like this in stable_baselines:

You need to redefine the policy completely for that, see https://stable-baselines.readthedocs.io/en/master/guide/custom_policy.html whereas SB3 supports that easily.

It is just that the stable_baslines3 does not support adding the NN graph to tensorboard, which was the main reason I discovered the mistake here

See https://stable-baselines3.readthedocs.io/en/master/guide/tensorboard.html#directly-accessing-the-summary-writer and https://github.com/DLR-RM/stable-baselines3/issues/286

[HadiSDev]

Alright