question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

BUG: Resample on PeriodIndex not working?

See original GitHub issue

import xarray as xr
import pandas as pd
da = xr.DataArray(pd.Series(1, pd.period_range('2000-1', '2000-12', freq='W')).rename_axis('date'))

da.resample('B', 'date', 'ffill')

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-1-eb64a66a8d1f> in <module>()
      3 da = xr.DataArray(pd.Series(1, pd.period_range('2000-1', '2000-12', freq='W')).rename_axis('date'))
      4
----> 5 da.resample('B', 'date', 'ffill')

/Users/maximilian/drive/workspace/xarray/xarray/core/common.py in resample(self, freq, dim, how, skipna, closed, label, base, keep_attrs)
    577         time_grouper = pd.TimeGrouper(freq=freq, how=how, closed=closed,
    578                                       label=label, base=base)
--> 579         gb = self.groupby_cls(self, group, grouper=time_grouper)
    580         if isinstance(how, basestring):
    581             f = getattr(gb, how)

/Users/maximilian/drive/workspace/xarray/xarray/core/groupby.py in __init__(self, obj, group, squeeze, grouper, bins, cut_kwargs)
    242                 raise ValueError('index must be monotonic for resampling')
    243             s = pd.Series(np.arange(index.size), index)
--> 244             first_items = s.groupby(grouper).first()
    245             if first_items.isnull().any():
    246                 full_index = first_items.index

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pandas/core/generic.py in groupby(self, by, axis, level, as_index, sort, group_keys, squeeze, **kwargs)
   3989         return groupby(self, by=by, axis=axis, level=level, as_index=as_index,
   3990                        sort=sort, group_keys=group_keys, squeeze=squeeze,
-> 3991                        **kwargs)
   3992
   3993     def asfreq(self, freq, method=None, how=None, normalize=False):

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pandas/core/groupby.py in groupby(obj, by, **kwds)
   1509         raise TypeError('invalid type: %s' % type(obj))
   1510
-> 1511     return klass(obj, by, **kwds)
   1512
   1513

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pandas/core/groupby.py in __init__(self, obj, keys, axis, level, grouper, exclusions, selection, as_index, sort, group_keys, squeeze, **kwargs)
    368                                                     level=level,
    369                                                     sort=sort,
--> 370                                                     mutated=self.mutated)
    371
    372         self.obj = obj

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pandas/core/groupby.py in _get_grouper(obj, key, axis, level, sort, mutated)
   2390     # a passed-in Grouper, directly convert
   2391     if isinstance(key, Grouper):
-> 2392         binner, grouper, obj = key._get_grouper(obj)
   2393         if key.key is None:
   2394             return grouper, [], obj

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pandas/tseries/resample.py in _get_grouper(self, obj)
   1059     def _get_grouper(self, obj):
   1060         # create the resampler and return our binner
-> 1061         r = self._get_resampler(obj)
   1062         r._set_binner()
   1063         return r.binner, r.grouper, r.obj

/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pandas/tseries/resample.py in _get_resampler(self, obj, kind)
   1055         raise TypeError("Only valid with DatetimeIndex, "
   1056                         "TimedeltaIndex or PeriodIndex, "
-> 1057                         "but got an instance of %r" % type(ax).__name__)
   1058
   1059     def _get_grouper(self, obj):

TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'Index'

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Comments:10 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
spencerkclarkcommented, May 22, 2018

+1 to this issue. I’m struggling big time with an 1800-year climate model dataset that I need to resample in order to make different annual means (June-May).

@lvankampenhout I agree that it would be nice if xarray had better support for PeriodIndexes.

Do you happen to be using a PeriodIndex because of pandas Timestamp-limitations? Despite the fact that generalized resample has not been implemented yet, I recommend you try using the new CFTimeIndex. As it turns out, for some one-off cases (like this one) resample is not too difficult to mimic using groupby. See the following example for your case. I’m assuming you’re looking for resampling with the 'AS-JUN' anchored offset?

from itertools import product
from cftime import DatetimeProlepticGregorian as datetime
import numpy as np
import xarray as xr

xr.set_options(enable_cftimeindex=True)

# Set up some example data indexed by cftime.DatetimeProlepticGregorian objects
dates = [datetime(year, month, 1) for year, month in  product(range(2, 5), range(1, 13))]
da = xr.DataArray(np.arange(len(dates)), coords=[dates], dims=['time'])
    
# Mimic resampling with the AS-JUN anchored offset
years = da.time.dt.year - (da.time.dt.month < 6)
da['AS-JUN'] = xr.DataArray([datetime(year, 6, 1) for year in years], coords=da.time.coords)
resampled = da.groupby('AS-JUN').mean('time').rename({'AS-JUN': 'time'})

This gives the following for resampled:

<xarray.DataArray (time: 4)>
array([  2. ,  10.5,  22.5,  32. ])
Coordinates:
  * time     (time) object 0001-06-01 00:00:00 0002-06-01 00:00:00 ...

This is analogous to using resample(time='AS-JUN') with a DataArray indexed by a DatetimeIndex:

import pandas as pd
dates = pd.date_range('2002-01-01', freq='M', periods=36)
da = xr.DataArray(np.arange(len(dates)), coords=[dates], dims='time')
resampled = da.resample(time='AS-JUN').mean('time')

which gives:

<xarray.DataArray (time: 4)>
array([  2. ,  10.5,  22.5,  32. ])
Coordinates:
  * time     (time) datetime64[ns] 2001-06-01 2002-06-01 2003-06-01 2004-06-01
0reactions
stale[bot]commented, Apr 28, 2020

In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity

If this issue remains relevant, please comment here or remove the stale label; otherwise it will be marked as closed automatically

Read more comments on GitHub >

github_iconTop Results From Across the Web

Pandas Resampling error: Only valid with DatetimeIndex or ...
When using panda's resample function on a DataFrame in order to convert tick data to OHLCV, a resampling error is encountered.
Read more >
v0.18.1.rst.txt - Pandas
has been enhanced to provide convenient syntax when working with ``.rolling(. ... ``PeriodIndex.resample`` where name not propagated (:issue:`12769`) - Bug ...
Read more >
Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex ...
This error happens when you use a pandas method which is only supposed to work with a datetime index. For instance, you might...
Read more >
Time series / date functionality - Pandas 中文
pandas contains extensive capabilities and features for working with time series data ... Resampling or converting a time series to a particular frequency....
Read more >
What's New — pandas 0.23.4 documentation
In the future pandas will not coerce, and the values not compare equal to the ... Indexing with a Boolean Index; PeriodIndex resampling;...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found