Working example code for R2D1

It seems like the code in experiments/ is outdated - I’ve managed to update the import errors and get most of the code running, however I get the following issue when running the atari_r2d1_gpu.py script:

Traceback (most recent call last):
  File "rlpyt/experiments/scripts/atari/dqn/train/atari_r2d1_gpu.py", line 50, in <module>
    build_and_train(*sys.argv[1:])
  File "rlpyt/experiments/scripts/atari/dqn/train/atari_r2d1_gpu.py", line 46, in build_and_train
    runner.train()
  File "/home/david/Repos/rlpyt/rlpyt/runners/minibatch_rl.py", line 240, in train
    opt_info = self.algo.optimize_agent(itr, samples)
  File "/home/david/Repos/rlpyt/rlpyt/algos/dqn/r2d1.py", line 133, in optimize_agent
    self.replay_buffer.append_samples(samples_to_buffer)
  File "/home/david/Repos/rlpyt/rlpyt/replays/sequence/prioritized.py", line 52, in append_samples
    T, idxs = super().append_samples(samples)
  File "/home/david/Repos/rlpyt/rlpyt/replays/frame.py", line 46, in append_samples
    buffer_samples = BufferSamples(*(v for k, v in samples.items()
TypeError: __new__() missing 2 required positional arguments: 'done' and 'prev_rnn_state'

Any tips on making the required modifications? If not, is there any place in the codebase which has working R2D1 example code?

[astooke]

Sorry! Was just a bad link, here it is:

https://github.com/astooke/rlpyt/blob/master/rlpyt/experiments/scripts/atari/dqn/launch/pabti/launch_atari_r2d1_async_alt_gravitar.py

[tesslerc]

@crizCraig , were you able to fix this? Tried a quick run here and it also hanged. Simple non async stuff (DQN) did work well. —edit— Seems I had some leftover python processes in the background after killing the program several times – killed them all, ran again and waited… it works 😃