Viewing results.

Thanks for putting together this library!

I have installed the library on a headless server, along with MongoDB and Visdom. Is there a way to view the results after running from the .yaml configs?

I am testing it out with the example

vel examples-configs/rl/atari/a2c/breakout_a2c.yaml train

Everything trains fine, but then when I look at the logfile at vel/output/openai/breakout_a2c/0/log.txt, it only saves the following:

Logging to /home/ygx/src/vel/output/openai/breakout_a2c/0

And the progress.csv at that directory level is empty.

When looking at the yaml config at https://github.com/yngtodd/vel/blob/master/examples-configs/rl/atari/a2c/breakout_a2c.yaml#L57, I see that it is saving a video. Where is that stored?

Thanks!

[MillionIntegrals]

Yes, that’s entirely possible but that requires to rework a bit evaluation command. Currently in the file evaluate_env_command.py I roll out environments and gather only the final reward:

    def record_take(self, model, env_instance, device, takenumber):
        frames = []

        observation = env_instance.reset()

        frames.append(env_instance.render('rgb_array'))

        print("Evaluating environment...")

        while True:
            observation_array = np.expand_dims(np.array(observation), axis=0)
            observation_tensor = torch.from_numpy(observation_array).to(device)
            actions = model.step(observation_tensor, **self.sample_args)['actions']

            observation, reward, done, epinfo = env_instance.step(actions.item())

            frames.append(env_instance.render('rgb_array'))

            if 'episode' in epinfo:
                # End of an episode
                return epinfo['episode']

You’d need to change the logic to gather rewards of each step, aggregate and then plot. To get more meaningful results you probably would also like to disable reward clipping when env is created which is used for training.

[yngtodd]

Just ran the most recent changes, and it works like a dream!

🔥 vel [master] 🍂  vel examples-configs/rl/atari/a2c/breakout_a2c.yaml evaluate
<ModelConfig at examples-configs/rl/atari/a2c/breakout_a2c.yaml>
================================================================================
Pytorch version: 0.4.1 cuda version 9.0.176 cudnn version 7102
Running model breakout_a2c, run 0 -- command evaluate -- device cuda
CUDA Device name Quadro P620
2018/11/13 - 16:52:51
================================================================================
Storage: <vel.storage.classic.ClassicStorage object at 0x7f025c08b4e0>
Evaluating environment...
Evaluating environment...
Evaluating environment...
Evaluating environment...
Evaluating environment...
       lengths     rewards
count      5.0    5.000000
mean   10001.0  399.000000
std        0.0   34.467376
min    10001.0  341.000000
25%    10001.0  399.000000
50%    10001.0  405.000000
75%    10001.0  425.000000
max    10001.0  425.000000
================================================================================
Done.
2018/11/13 - 16:54:40
================================================================================

Thank you for taking the time to help me out!