[Question] Where to find the definitions of the terms shown in training?

Important Note: We do not do technical support, nor consulting and don’t answer personal questions per email. Please post your question on the RL Discord, Reddit or Stack Overflow in that case.

Question

I was looking for documentation for definitions of the terms used in training, e.g.

-----------------------------------------
| time/                   |             |
|    fps                  | 2447        |
|    iterations           | 3           |
|    time_elapsed         | 30          |
|    total_timesteps      | 73728       |
| train/                  |             |
|    approx_kl            | 0.012046688 |
|    clip_fraction        | 0.124       |
|    clip_range           | 0.2         |
|    entropy_loss         | -2.28       |
|    explained_variance   | 0.58        |
|    learning_rate        | 0.0003      |
|    loss                 | 0.31        |
|    n_updates            | 20          |
|    policy_gradient_loss | -0.0174     |
|    value_loss           | 1.39        |
-----------------------------------------

I could not find anything. As an example for value_loss all I got was a link to the source code for PPO which says

# Value loss using the TD(gae_lambda) target
value_loss = F.mse_loss(rollout_data.returns, values_pred)

This gives a clue but doesn’t tell me the whole story. Is there any documentation for these terms I am missing?

[araffin]

probably a duplicate of #662 , PR is welcomed 😉

[duttashi]

@Miffyli I agree and I feel its very important to have elaborate documentation related to the results generated by algorithms implemented in SB3. Absence of which may render fewer users of SB3 library. For instance, consider the Scikit-learn library for machine learning. One reason for its popularity is the extensive documentation pertaining to algorithms in code format. I’ve implemented PPO in SB3 and I’ve no idea how to interpret the results. Its like black magic and honestly does not go in good taste, if I were to use it in production!