Framewise transcription evaluation

TL;DR: Basically all I’m asking for is taking frames as inputs to @rabitt’s mir_eval.multipitch module.

Hi everyone,

recent transcription papers:

• Kelz et al., "On the Potential of Simple Framewise Approaches to Piano Transcription ". https://arxiv.org/abs/1612.05153
• Sigtia et al., “An End-to-End Neural Network for Polyphonic Piano Music Transcription”

This is basically http://scikit-learn.org/stable/modules/model_evaluation.html#multiclass-and-multilabel-classification with using the macro samples parameter, but scaled with the number of frames labels.

As it seems that people use it, would it be useful to have this in mir_eval? If we go with the scikit-learn implementation which I would strongly suggest, this adds it back as a dependency. Opinions?

[justinsalamon]

Btw, note-level eval is already implemented in mir_eval, as is multi-f0. So (assuming the implementations are correct) to get frame-level metrics for transcription you’d just have to sample the note events onto a fixed time grid (which I think is also already implemented somewhere) and then feed that into the multi-f0 metrics, as @stefan-balke noted in the first comment. Pinging @rabitt

[stefan-balke]

Yep, on my list. @justinsalamon, see you at ICASSP then 😃