Make partial_fit force a single iteration

Currently partial_fit and fit + warm_start have the same behavior. I think it would be more useful if we followed SGD’s example and trained for exactly one epoch when partial_fit is called, regardless of the value of epochs/n_iter passed to __init__.

[stsievert]

Yup. I’m working on a PR for that right now, and it’s pretty easy.

[stsievert]

The workaround to making [SciKeras] work correctly with HyperbandSearchCV would be not to specify the epochs in the initialization so that epochs=1.

Yes. That’s correct. SciKeras should use epochs=1 to fit inside of HyperbandSearchCV’s rule of thumb. I would probably specify this manually because I don’t like relying on defaults.

Of course, epochs=2 will not throw any errors, but it will require a small modification of the rule of thumb.

Does that answer your question?

In the current implementation …

I think you’re largely correct. Here’s what happens right now:

• HyperbandSearchCV’s treats one iteration in max_iter as one partial_fit call.*
• In most Scikit-learn models, one partial_fit call corresponds to one pass through the dataset (e.g, MLPRegressor, SGDRegressor).
• In the current implementation of SciKeras, partial_fit and fit perform the same action and train for all epochs, not a single pass through the dataset.

^{*As detailed in Hyperband’s rule of thumb, the number of examples for each call should vary}