confusing performance_metrics

Hi,

I guess I am having the same issues as others ( #941 ?) to understand performance_metrics

What one would typically expect to have when doing k-CV is k values , for each validation metric. In the case of prophet, if I understood correctly, k = # of cutoffs (which should be also k = int( (df.shape[0] - initial - horizon) / period) + 1 ) (just as double check - can you confirm?)

• how can I get a df_p (performance_metrics) whose size = # cutoffs?

• the confusion arises also from the fact that you are probably using term horizon with two different meanings: in the docs at the beginning it seems to be a fixed window size, while after, it becomes a column with many values…

Thanks

[ggrrll]

so, indeed, CV in time series is more tricky …

here is a nice post describing different possible approaches ( I guess the main ones – I am not a big expert) https://medium.com/@soumyachess1496/cross-validation-in-time-series-566ae4981ce4

I guess the devs here (@bletham , …) could think of integrating few other CV techniques in the future (like the Blocked CV, which is what @ivangvi basically proposed, if I understood…)

[ofwallaart]

Great answers guys! Much appreciated, as often in this field of science the answer is: it depends. Very interesting to read about your considerations and trade-offs.