Clarify which policies share weights between policy and value network.

The documentation does not state explicitly whether the default policies in common.policies share weights between the value network and the policy network. After carefully reading the code, I could deduce that the

• LstmPolicy shares all weights between value and policy network except for the very last linear layer.
• FeedForwardPolicy shares weights if the ‘cnn’ extractor is used but uses two entirely different data streams if the ‘mlp’ extractor is used.

Are there any justifications for this specific setup?

[ernestum]

Uhhh I can hear a PR rumbling in the distance …

[ernestum]

I like the intermediary approach. It takes away the option to “rejoin” the two data streams which was probably useless in the first place. I might start to work on it when I get around to it.