[rllib] How can I record the full episode results of rllib?

What is your question?

Ray version and other system information (Python version, TensorFlow version, OS):

Python version: Python 3.6.10 :: Anaconda, Inc.
Anaconda version: 4.7.12
TensorFlow version: 1.15.0
OS: Ubuntu 16.04
ray version: 0.8.1

Hello I’ve trained an RL model using rllib I have tested the breakout environment and the agent is running successfully.

Here is the python code I used. I used Rllib’s python API If you test at openai gym after learning, it works successfully.

I want to record the results of the agent. Agent test was executed using the rollout command of rllib. My result photo is below. This is the command I executed

rllib rollout checkpoint_4301/checkpoint-4301 \
    --run PPO --env BreakoutNoFrameskip-v4 --monitor --config '{"monitor": true}'

When testing with rollout, on average 8 games run. Many episodes ran, but only the results of one game (four lives in the case of breakout) were saved as videos. And I think it’s a mix of different game parts.

I additionally tested ms pacman. Pacman’s recording showed the same problem.

I have attached my rllib model and the recorded results file. I would be grateful if you could tell me a document or a way to help me record the video.

PPO_BreakoutNoFrameskip-v4_2020-02-11_21-32-52m48o1b48.zip BreakoutModel.zip

[sven1977]

I can reproduce the issue with CartPole as well. It’s only recording a very short sequence of the rolled out episode, and only a few of these as well (4 out of 50 rolled out episodes have mpeg snippet files). Taking a look. …

[sven1977]

Ok, checkout this PR and let me know, whether this fixes your problems: https://github.com/ray-project/ray/pull/7347

I separated the --monitor option from the --out option (which used to be silently(!) required to record any videos into the --out + "monitor" directory).

The new command line would be (tested on CartPole with episodes of 1000 ts): rllib rollout [your checkpoint file] --run PPO --env BreakoutNoFrameskip-v4 --video-dir [some dir where all(!) episode videos will be stored in full length]