Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Achieving reported training performance

See original GitHub issue

We trained a few agents with the training code provided in repo. If don’t change anything the mean reward in 500 training levels in Starpilot game is around 5.6. If we remove the VecNormalize line in environment creation, then we achieve mean reward around 9.2. However the reported mean reward is around 12 in the paper (Figure 4).

Did you use VecNormalize in the paper?

Issue Analytics

State:
Created 3 years ago
Comments:9 (2 by maintainers)

Top GitHub Comments

1reaction

hfenisercommented, Jun 3, 2020

We are able to achieve up to 18.7 mean reward in starpilot game by introducing 8 initial states.

EDIT: We achieved 18.7 at 400M training with 8 initial states. In 200M timesteps the best we achieved is 14.9 and the average is 14.16. Sorry for the inconvenience.

1reaction

hfenisercommented, May 20, 2020

We achieved 13.9 reward in starpilot game by running 1 worker with 256 env with VecNormalize. Therefore I am closing this issue.

Top Results From Across the Web

How to Measure Training Effectiveness in 2023? | Simplilearn

Know how to measure training effectiveness and learn proven methods to ensure that the original learning goals and purpose are achieved.

Achieving performance in the training industry

Feedback and measurement demonstrate the impact training has on employee performance. We can track training status and impact through real-time ...

Getting more from your training programs - McKinsey

To improve results from training programs, executives must focus on what happens in the workplace before and after employees go to class.

Performance Training - Overview, How It Works, Benefits

Performance training is a training strategy that assists participants in achieving their targeted performance goals and/or objectives.

7 Ways To Ensure Training Results Translate to Business ...

One critical factor for performance success is feedback. Practice makes permanent, but feedback makes perfect. Coaches can use feedback to address application ...