RFC Consistent options/names for loss and criterion

Proposal

Make the options to choose a loss or a split criterion consistent/unified over all estimators, classifiers as well as regressors.

Current State of Affairs

The API for setting a loss function or a split criterion is scattered. Here is an incomplete list as of v0.23.0:

Classifiers

• sklearn.ensemble.GradientBoostingClassifier
  • loss : {‘deviance’, ‘exponential’}, default=’deviance’
  • criterion : {‘friedman_mse’, ‘mse’, ‘mae’}, default=’friedman_mse’
• sklearn.ensemble.HistGradientBoostingClassifier
  • loss : {‘auto’, ‘binary_crossentropy’, ‘categorical_crossentropy’}, optional (default=’auto’)
• sklearn.linear_model.SGDClassifier
  • loss : {‘hinge’, ‘log’, ‘modified_huber’, ‘squared_hinge’, ‘perceptron’, ‘squared_loss’, ‘huber’, ‘epsilon_insensitive’, ‘squared_epsilon_insensitive’.}, default=’hinge’
• sklearn.svm.LinearSVC
  • loss : {‘hinge’, ‘squared_hinge’}, default=’squared_hinge’
• sklearn.tree.DecisionTreeClassifier
  • criterion : {“gini”, “entropy”}, default=”gini”

Regressors

• sklearn.ensemble.GradientBoostingRegressor

  • loss : {‘ls’, ‘lad’, ‘huber’, ‘quantile’}, default=’ls’
  • criterion : {‘friedman_mse’, ‘mse’, ‘mae’}, default=’friedman_mse’

• sklearn.ensemble.HistGradientBoostingRegressor

  • loss : {‘least_squares’, ‘least_absolute_deviation’, ‘poisson’}, optional (default=’least_squares’)

• sklearn.linear_model.SGDRegressor

  • loss : {‘squared_loss’, ‘huber’, ‘epsilon_insensitive’, ‘squared_epsilon_insensitive’}, default=’squared_loss’

• sklearn.tree.DecisionTreeRegressor

  • criterion : {“mse”, “friedman_mse”, “mae”}, default=”mse”

• sklearn.svm.LinearSVR

  • loss : {‘epsilon_insensitive’, ‘squared_epsilon_insensitive’}, default=’epsilon_insensitive’

Additional context

Supersedes #3481. Furthermore, #15123 is a bit related, though that one is about private functionality while this one is about public API.

Update

• "squared_error" in #19310
• "absolute_error in #19733
• Decide name for log loss
  • #23036
  • #23040
  • #23046
  • #23047

[lorentzenchr]

🚀 An RFC issue that gets closed! That does not happen every day. Thanks to everyone involved! 🚀

[agramfort]

+1 for “Maybe we could make an exception for tree-based models and accept both names (as aliases)?”

and clarify the docstring

Message ID: @.***>