Canonical Channels First/Last Clarification

📚 Documentation

This is a very minor quibble with in-code documentation/defaults, but: I noticed that is_image_space has a default channels_last=True parameter, even though in NatureCNN, the assumption is that channels are first.

In NatureCNN, is_image_space isn’t called with check_channels, so that parameter doesn’t end up being used, but I briefly found it a bit confusing, when I was debugging my image spaces being the right construction, that the default value for channels_last in the function definition conflicted with the assumed default for the canonical CNN feature extractor.

I realize that switching the default value on is_image_space would be a pain, since it would be a breaking change to peoples’ code, but perhaps a workable middle ground would be to just add a clarification to the docstring of channels_last here saying something like “by the way, this parameter doesn’t come into play unless check_channels is set, and also the default being True is different from NatureCNN’s expectations”

### Checklist

• [x ] I have read the documentation (required)
• [x ] I have checked that there is no similar issue in the repo (required)

[Miffyli]

(Unrelated to this issue, I had actually thinking of checking in with the maintainers to see if there was any interest in taking as a PR any of the aspects of how I chose to design support for Dict Action spaces in our codebase, which revolves around a MultiModalActionDistribution class that that provides the external interface of a single action distribution, but internally can maintain multiple different distribution types, and then merge those together as is appropriate for sampling, entropy, log prob, etc. No worries if your goal is to go down a different design route, but since our project had independent reason to do the work, it seemed at least worth seeing if it’d something you’d want to integrate)

We would definitely be interested in having support for more complex action spaces down the line! However as there does not seem to be “correct” approach to it (how to handle dependencies etc. See this link shared by @arrafin ), it would be better to start with a discussion on contrib first and see how much can be done there before touching SB3 too much.

[decodyng]

I see, maybe another thing to do (in addition to the doc PR) would be to explicitly specify check_channels=False in the NatureCNN to avoid confusion?

Yeah, I think that would be a nice additional bit of clarification!

For dict observations, you might have a look at #243, it should be merged with master once #351 is ready. (i’ve updated the env checker there too). Dict action spaces are much more tricky to handle, so the support is not really planned yet.

Cool, good to know! I’ve already written the code I need to make both sides of this work (and amusingly/frustrating possibly ended up reinventing the wheel relative to what you guys did, since the bulk of it was written in its SB2 form before that PR got going) but it definitely would be nice in the long run for it to be fully supported by SB3 itself.

(Unrelated to this issue, I had actually thinking of checking in with the maintainers to see if there was any interest in taking as a PR any of the aspects of how I chose to design support for Dict Action spaces in our codebase, which revolves around a MultiModalActionDistribution class that that provides the external interface of a single action distribution, but internally can maintain multiple different distribution types, and then merge those together as is appropriate for sampling, entropy, log prob, etc. No worries if your goal is to go down a different design route, but since our project had independent reason to do the work, it seemed at least worth seeing if it’d something you’d want to integrate)