Save training and vaildation loss in `loss_curve_` in MLPClassifier and MLPRegressor with early_stopping

Let’s select MLPClassifier.

In MLPClassifier there is loss_curve_ available. If there is early_stopping enabled then some part of the data is used as validation. Can we save the loss of training and validation data in the loss_curve_ as well?

Additional context

I’ve compared the MLP implementation with Tensorflow implementation and it works very well, there are no significant differences in the performance. You can read the comparison details at my blog post. I’m using the MLP in my AutoML mljar-supervised which creates Markdown reports for each model. I would like to have learning curves available in the report.

[glemaitre]

Having callbacks will be super helpful. Do you know when they will be available?

I personally did not follow the advancement of this feature and if there is any blocker regarding the API. It will be probably too soon to expect it in 0.24 since we should release pretty soon but it might be a milestone for 0.25 then.

[pplonski]

That’s a strange decision, sklearn MLP works pretty well. I did a comparison of MLP from sklearn vs Keras+TF. Sklearn MLP performs very well and was faster on CPU computations. Check the comparison here: https://mljar.com/blog/tensorflow-vs-scikit-learn/ Not all NN must be deep on computed on GPU.