Cannot cast array data from dtype('<M8[ns]') to dtype('float64') according to the rule 'safe'

While trying to forecast predictions and plot the confidence intervals, I received the following error: Cannot cast array data from dtype('<M8[ns]') to dtype('float64') according to the rule 'safe'. I’m not sure what’s going on, as in my code I also transformed the index datetime dtype into a float manually. Is this related to some other error in my code and/or setup? My notebook is available here: https://drive.google.com/file/d/1B7kmDmdqY-zYFscL-LyV2nTJd_GGfAJ1/view?usp=sharing. The code uses public data and loads it via a request call, so you should be able to run it as is. There are no external dependencies beyond those used in the tutorial.

This is especially odd to me because I confirmed that the datetime type (datetime64[ns]) is the same for my dataset as for the dataset used in the example before I transform it to a float (it fails either way):

My data

>>> # Assess Index dtype
>>> daily_covid_df.index
DatetimeIndex(['2020-01-22', '2020-01-22', '2020-01-23', '2020-01-23',
               '2020-01-24', '2020-01-24', '2020-01-25', '2020-01-25',
               '2020-01-26', '2020-01-26',
               ...
               '2020-09-11', '2020-09-11', '2020-09-11', '2020-09-11',
               '2020-09-11', '2020-09-11', '2020-09-11', '2020-09-11',
               '2020-09-11', '2020-09-11'],
              dtype='datetime64[ns]', name='date', length=10738, freq=None)

Your data

>>> import pandas as pd
>>> url = "https://raw.githubusercontent.com/numenta/NAB/master/data/realTweets/Twitter_volume_AMZN.csv"
>>> df = pd.read_csv(url, header=0, index_col=0, parse_dates=True)
>>> df.index
DatetimeIndex(['2015-02-26 21:42:53', '2015-02-26 21:47:53',
               '2015-02-26 21:52:53', '2015-02-26 21:57:53',
               '2015-02-26 22:02:53', '2015-02-26 22:07:53',
               '2015-02-26 22:12:53', '2015-02-26 22:17:53',
               '2015-02-26 22:22:53', '2015-02-26 22:27:53',
               ...
               '2015-04-22 20:07:53', '2015-04-22 20:12:53',
               '2015-04-22 20:17:53', '2015-04-22 20:22:53',
               '2015-04-22 20:27:53', '2015-04-22 20:32:53',
               '2015-04-22 20:37:53', '2015-04-22 20:42:53',
               '2015-04-22 20:47:53', '2015-04-22 20:52:53'],
              dtype='datetime64[ns]', name='timestamp', length=15831, freq=None)

Also, I’m not sure if I understand the logic of what is done when the test_data object is created.

# Create Test Split
test_data = ListDataset(
    [{"start": example_ny_df.index[0], "target": example_ny_df.positiveIncrease}],
    freq = FREQ
)

I would have presumed that because the start is the earliest date in the dataset and the target is the full data up through and to last date, that the plotting code for

for test_entry, forecast in zip(test_data, predictor.predict(test_data)):
    to_pandas(test_entry)[-60:].plot(linewidth=2)
    forecast.plot(color='g', prediction_intervals=[50.0, 90.0])
plt.grid(which='both')

would plot predictions from the start date to the end date; however, in my code it plots from mid-August to the first week of September. Is this because of the -60 in the to_pandas(test_entry)[-60] part?

Is there documentation available that explains these functions a bit more or should I just reference the code itself?

Thanks for your time and attention!

[AlexMRuch]

Forgot to thank you for this advice and to let you know that getting the different accuracy metrics worked well for my model pipeline. Unfortunately, though, I’ve switched to using another library that is a bit easier to use and is more flexible for my modeling needs right now (https://github.com/jdb78/pytorch-forecasting). I hope the notes above help others who wish to use pytorch-ts for forecasting! Thanks again for your time and for helping me test pytorch-ts. I look forward to watching it as it develops!

[kashif]

So the multivariate grouper is just a helper function to make multivariate time series out of the open datasets which are all uni-variate. If you prepare the dataset yourself then you can just make it via ListDataset but now the target will be an 2d-array and you will need to set the flag one_dim_target=False.