[ENH] Behaviour modifying compositors for forecasting

Is your feature request related to a problem? Please describe. In the Feb 2 meeting it was discussed that some behaviours in evaluation might actually be “business of the model” rather than of the evaluation. Irrespectively, even if evaluation workflows implement some of these behaviours, it might also be useful to have certain rules as behaviour modifying compositors for forecasters.

Describe the solution you’d like It might be useful to implement wrapper compositors that modify the behaviour of fitting/updating:

• “sliding window update modifier” - wraps a Forecaster and replaces update of the wrapped estimator with a sliding window re-fit. Hyper-parameters are sliding window parameters.
• “re-fit after T time modifier” - wraps a Forecaster and replaces update with the following meta-algorithm: if time since last fit is <T, then call update of wrapped forecaster. Otherwise, re-fit to a pre-specified window (fixed length or all past). Hyper-parameters are T and the window specs.
• “re-fit once forecast error is too large” - wraps a Forecaster and replaces update with the following meta-algorihtm: re-fits if forecast error, evaluated in a certain way, exceeds a threshold; otherwise calls the wrapped update. To check the condition, internally calls the evaluator. Hyper-parameters are: evaluation parameters, including loss metric; threshold that needs to be passed for re-fit to be triggered. Could be dynamic, so might be interesting to think about how to make this more abstract to allow arbitrary smoothers/anomaly detectors on the loss metric.

Describe alternatives you’ve considered The alternative would be to have “merged/contracted” estimators or composites in cases where these compositors might be applied instead, e.g., as hyper-parameter options of grid search tuners.

[ZiyaoWei]

I’m interested in trying to do this if it’s still relevant - I assume this would be implementing a couple new forecasters in the same sktime.forecasting.stream package?

[fkiraly]

@mloning, yes, your idea sounds like an improvement over mine. Perhaps one “conditional refitter (simple)” and one “pipeline conditional refitter (adtk style)”?