Passing parameters to the policy is hard

I am doing a hyperparameter sweep and therefore would like to parameterize the network size of my policy. As I understood, to change the size of the policy network, I have to derive my own policy class like here. If I want to parameterize the size of the network, I have to use a weird construct like this:

def make_model(units_per_layer, num_layers, *args, **kwargs):
    class MyCustomMLP(FeedForwardPolicy):
        def __init__(self, *args, **kwargs):
            super(MyCustomMLP, self).__init__(*args, **kwargs,
                                              layers=[units_per_layer] * num_layers,
                                              feature_extraction="mlp")
    return PPO2(MyCustomMLP, *args, **kwargs)

It gets even more weird when I want to load a model because I need to somehow externally figure out the size of the network before I load it.

What would you consider the most elegant way to solve this issue?

If there was a way to pass custom arguments to the policy constructor that would help a bit, but would not solve the persistence problem right away. Passing a factory function for the policy instead of a policy class also does not help because in PPO2 there is a check if the policy is really an ActorCriticPolicy.

Any input is highly appreciated.

Edit: I just saw that the issue is partially addressed in the pull requests #83 and #84 but they do not seem to solve the store/load issue.

[ernestum]

Looks like @hill-a did not start working on this one. So I will now.

[araffin]

Note: we should address this issue once #91 is solved.