Additional metrics in `sklearn.metrics.classification_report`

Describe the workflow you want to enable

Metrics are Extremely Important for benchmarking models’ performance. But in scikit-learn, it is not easy to extract these metrics from a multiclass classification model. In the past, I had to combine different pieces of functions within the sklearn.metrics module in order to derive my own. But it is counterproductive, so I end up staying away from scikit-learn, and using other libraries such as pycm, mlxtend, yellowbrick, … etc to get the job done. It is not perfect and still requires some customization, but they are more complete, and much easier to use than scikit-learn. So it would be great if scikit-learn can improve a little more in this area, so that we can focus more on modeling than customizing the code.

Describe your proposed solution

metrics.classification_report is a good start. The current metrics.classification_report return

• Precision (Positive Predictive Value)
• Recall (True Positive Rate)
• F1
• Support

It is great, but far from complete. The following 2 are very important as well

• Negative Predictive Value (NPV)
• Specificity (True Negative Rate, or TNR)

Once we can cover these 4 metrics for the multiclass classification

• Precision (Positive Predictive Value) --> requires (TP, FP)
• Recall (True Positive Rate) --> requires (TP, FN)
• Negative Predictive Value (NPV) --> requires (TN, FN)
• Specificity (True Negative Rate) --> requires (TN, FP)

we can pretty much derive the rest of the metrics, such as

• FNR (False Negative Rate = 1 - TPR)
• FPR (False Positive Rate = 1 - TNR)
• FDR (False Discovery Rate = 1 - PPV)
• FOR (False Omission Rate = 1 - NPV)
• ACC (Accuracy)
• MCC (Mathews Correlation Coefficient)
• Prevalence Threshold
• …etc

These 2 are quite important as well, but it can be tricky to get for multiclass classification

• AUROC (Area under ROC)
• AUPR (Area under Precision-Recall curve)

The pycm library is quite comprehensive. Perhaps you may consider integrating some of their goodies into scikit-learn.

Describe alternatives you’ve considered, if relevant

No response

Additional context

No response

[adrinjalali]

@glemaitre would you mind creating an issue which would cover all-ish the open issues we have and have your proposed APIs there, so that the rest of us can have a look and take the API discussion from there?

[glemaitre]

I would advise not to start working on this issue since that there are no feedbacks from @scikit-learn/core-devs regarding the API