AtariWrapper does not use recommended defaults

The current AtariWrapper by default has terminate_on_life_loss set to True. This goes against the recommendations of Revisiting the Arcade Learning Environment (https://arxiv.org/pdf/1709.06009.pdf). I believe this should be set to False by default. They also recommend using sticky actions instead of noop resets, but I think that problem is outside the scope of this wrapper.

[jkterry1]

My concern, and I’m sure @JesseFarebro (the maintainer of the ALE) would agree, is that the settings in Gym environments for Atari was never really done to begin with and that for people doing future work with then should use what have been the recommended practices with then for years. This actually caused an issue with us working with Atari games for an ICML paper, which is why Ryan created the issue.

[RyanNavillus]

I just realized that I should have put this issue in the actual stable baselines 3 repo, but I guess it’s relevant here as well. I definitely understand the trade-off between using newer recommendations and preserving fair comparisons to previous work.