Support early_stopping with custom validation_set

Describe the workflow you want to enable

Today in SGDClassifier, the parameter early_stopping uses a fraction of the data randomly, it would be useful to support a custom validation set chosen by the user.

Describe your proposed solution

for example:

clf = SGDClassifier(early_stopping=True)
clf.fit(X_train, y_train, eval_set=(X_val, y_val))

[ogrisel]

Note that auto-splitting from the training set inside the final classifier/regressor is problematic when this estimator is wrapped in a rebalancing meta-estimator to tackle target imbalance problems: rebalancing should happen only on training data while early stopping, model selection and evaluation should only use metrics computed using originally balanced data.

I am not sure an auto-magical API would work for this. Making it possible to pass a manually prepared validation might be the sanest way to deal with this situation.

[adrinjalali]

I guess we could think of an API where pipeline’s fit accepts a validation_set, as well as many other estimators and all other meta-estimators, and handle that properly. But that’s a quite a large project to pull off.