Support Tuple and Dict spaces

If the obs_space of an environment is a spaces.Tuple, then DummyVecEnc breaks.

class DummyVecEnv(VecEnv):
    def __init__(self, env_fns):
        self.envs = [fn() for fn in env_fns]
        env = self.envs[0]
        VecEnv.__init__(self, len(env_fns), env.observation_space, env.action_space)
        obs_space = env.observation_space
 
        self.keys, shapes, dtypes = obs_space_info(obs_space)       
        self.buf_obs = { k: np.zeros((self.num_envs,) + tuple(shapes[k]), dtype=dtypes[k]) for k in self.keys }

This breaks because self.keys is None. This happens when obs_space is spaces.Tuple, given that spaces.Tuple is not instance of spaces.Dict, and therefore obs_space_info returns [None], {None: None}, {None: None}. So, I’m not sure how to use the DummyVecEnv now…

[Atcold]

Alright, spaces.Dict are also not supported. I’ve patched it. If anyone is interested, here’s my patch.

https://github.com/Atcold/baselines/commit/dbc329f28c968fc261f53ebafc9f53192caf967c

[joshim5]

https://github.com/openai/baselines/pull/911 patches this for Tuples