Icevision records metrics as a dict; in fastai, a scalar would provide better integration

In FastAI, various callbacks can access and use recorded metrics. For example, the SaveModelCallback can save the model in the case that it is better than the previous best. It looks a the values in the learner’s recorder for this information (learn.recorder.values). Normally, it expects to see a scalar value, but icevision appears to stash a dict there.

For example, when using the cocometrics, learn.recorder.values[0][2] is a dict with a single key - {‘mAPI’ : 0.1234}. If the metrics were stashed directly, then the additional machinery would work. This would require a little additional adaptation to provide the metric name as well.

This callback can be used as-is, but only with valid_loss, training_loss rather than the metric (which is better aligned with the model’s task performance).

[FraPochetti]

As discussed on Discord, I think this would be a better match for me right now! Just for sake of clarity, I will unassign myself.

[FraPochetti]

Ok, will keep this open then.