Some questions about the network layer of parking env (SAC+HER)

Hi, i tried to run the parking env and got the success rate of over 95%. But I still have some questions about the model structure. The model as follows :

model = HER('MlpPolicy', env, SAC, n_sampled_goal=4,
            goal_selection_strategy='future',
            verbose=1, buffer_size=int(1e6),
            learning_rate=1e-3, gamma=0.9, batch_size=256,
            policy_kwargs=dict(layers=[256, 256, 256]))          

I’m not really understanding about policy_kwargs. I think it is the parameter to setup model or create policy and target TF objects, but i am not sure. This model does not only have one network makes me confuse. May i ask you about the neural network architecture of this env ? If you have time to answer, I hope to figure out how many layers are there and how many neurons in each layer.

Looking forward to your reply, thanks.

[eleurent]

No, for this particular task I already had satisfying results with Mlp so I did not try more sophisticated models (plus the observation is quite small and unstructured).

For the intersection environment however, I compared MLP with CNN, as well as an attention-based model, see paper and results (with a runnable colab notebook).

[eleurent]

env.observation_space defines the inputs of the policy, while env.action_space defines its outputs.

• env.observation_space is typically a continuous Box, and you can get its dimension by calling env.observation_space.shape
• env.action_space can be a Box, but also a Discrete space, in which case you can get its size by calling env.action_space.n