[BUG] Gridsearch in Ensemble throws a ValueError : Cannot instantiate EnsembleModel with trained/fitted models. Consider resetting all models with `my_model.untrained_model()`

Trying to fit a gridsearch ensemble, throws a Value Error.

To Reproduce import time t_start1 = time.perf_counter() import sys import numbers import time import math import pandas as pd import numpy as np from numpy import array import datetime as dt from functools import reduce import datetime from datetime import datetime import statsmodels.api as sm from scipy.stats import normaltest from darts import TimeSeries from darts.models import ( NaiveSeasonal, NaiveDrift, Prophet, ExponentialSmoothing, ARIMA, AutoARIMA, Theta, BlockRNNModel, RegressionEnsembleModel ) from darts.metrics import mape, mase, mae, mse, ope, r2_score, rmse, rmsle from darts.utils.statistics import check_seasonality, plot_acf, plot_residuals_analysis from darts.dataprocessing.transformers.boxcox import BoxCox import datetime import numpy as np from collections import OrderedDict from openpyxl import load_workbook import string import itertools import statsmodels.api as sm

#Creating a randomly generated timeseries N = 90 rng = pd.date_range(‘2019-01-01’, freq=‘3D’, periods=N) df = pd.DataFrame(np.random.rand(N, 1), columns=[‘temp’], index=rng) series = TimeSeries.from_dataframe(df)

#Training and testing train = series[:72] val = series[-18:] #Models - I have tried .untrained_model() as well. (same error) m_expon = ExponentialSmoothing() m_naive = NaiveDrift() m_arima = AutoARIMA() m_prophet = Prophet()

models = [ m_expon,
m_arima, m_naive, m_prophet]

n_train = len(train)-30
a_list = list(range(1,5)) a_array = np.array(a_list)

parameters={ ‘forecasting_models’:[models], ‘regression_train_n_points’: a_array.tolist() }

ensemble_model = RegressionEnsembleModel.gridsearch(parameters, series=train, stride = 1, forecast_horizon=2, verbose=False, n_jobs = 1)

Expected behavior Should run without throwing valueError

**ERROR **

ValueError: Cannot instantiate EnsembleModel with trained/fitted models. Consider resetting all models with my_model.untrained_model()

• Python version: 3.7
• darts version 0.17.1

I’m not sure if I am missing something please feel free to let me know. Thanks !

[dennisbader]

Thanks for pointing this out.

The issue is that after the the first iteration of gridsearch parameters, forecasting_models have been trained and are not getting reset for the second set of parameters.

We will have to fix that.

In the meantime you could implement something similar by looping over a_array and by creating a new RegressionEnsemble model istance each iteration. You will have to use resetted forecasting_models at model creation like below:

for a in a_array:
    my_ensemble_model = RegressionEnsembleModel(
        forecasting_models=[model.untrained_model() for model in models],
        regression_train_n_points=a
    )
    # compute something for best parameter selection

Using my_ensemble_model.backtest() you could find for which element in a_array the model performs best.

[Shubha1m]

Figured out the issue.

This works !