For some objectives where baseline was 0, "pct better than baseline" is nan

{'F1': nan,
 'MCC Binary': nan,
 'Log Loss Binary': 93.29789549298991,
 'AUC': 58.36492736629537,
 'Precision': nan,
 'Balanced Accuracy Binary': 63.46659876071641,
 'Accuracy Binary': 12.876088314169193}

I’ve created a Jupyter notebook that reproduces this problem in evalml, and attached it and the associated datafile to a thread in Slack.

[rpeck]

Like. 😃

[rpeck]

Further: I agree with the decision. IMO, if a metric is [usually, at least] 0…1, then going from 0 to 0.2 feels like a 20% improvement, even though mathematically it isn’t. In a way, this reminds me of all of those formulas that take the log of a quantity, but they add 1 first so that they don’t take the log of 0. 🙂