Decode time error in CESM POP output

Hi, all

Recently, I found a error about time decoding. The .nc file is POP output.

In [29]: ds = xr.open_mfdataset('EXAMPLE_CASE.pop.h.0001-01.nc')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-29-576c2580c841> in <module>()
----> 1 ds = xr.open_mfdataset('EXAMPLE_CASE.pop.h.0001-01.nc')

/Users/Yefee/miniconda2/lib/python2.7/site-packages/xarray/backends/api.pyc in open_mfdataset(paths, chunks, concat_dim, preprocess, engine, lock, **kwargs)
    300         lock = _default_lock(paths[0], engine)
    301     datasets = [open_dataset(p, engine=engine, chunks=chunks or {}, lock=lock,
--> 302                              **kwargs) for p in paths]
    303     file_objs = [ds._file_obj for ds in datasets]
    304

/Users/Yefee/miniconda2/lib/python2.7/site-packages/xarray/backends/api.pyc in open_dataset(filename_or_obj, group, decode_cf, mask_and_scale, decode_times, concat_characters, decode_coords, engine, chunks, lock, drop_variables)
    225             lock = _default_lock(filename_or_obj, engine)
    226         with close_on_error(store):
--> 227             return maybe_decode_store(store, lock)
    228     else:
    229         if engine is not None and engine != 'scipy':

/Users/Yefee/miniconda2/lib/python2.7/site-packages/xarray/backends/api.pyc in maybe_decode_store(store, lock)
    156             store, mask_and_scale=mask_and_scale, decode_times=decode_times,
    157             concat_characters=concat_characters, decode_coords=decode_coords,
--> 158             drop_variables=drop_variables)
    159
    160         if chunks is not None:

/Users/Yefee/miniconda2/lib/python2.7/site-packages/xarray/conventions.pyc in decode_cf(obj, concat_characters, mask_and_scale, decode_times, decode_coords, drop_variables)
    888     vars, attrs, coord_names = decode_cf_variables(
    889         vars, attrs, concat_characters, mask_and_scale, decode_times,
--> 890         decode_coords, drop_variables=drop_variables)
    891     ds = Dataset(vars, attrs=attrs)
    892     ds = ds.set_coords(coord_names.union(extra_coords))

/Users/Yefee/miniconda2/lib/python2.7/site-packages/xarray/conventions.pyc in decode_cf_variables(variables, attributes, concat_characters, mask_and_scale, decode_times, decode_coords, drop_variables)
    823         new_vars[k] = decode_cf_variable(
    824             v, concat_characters=concat, mask_and_scale=mask_and_scale,
--> 825             decode_times=decode_times)
    826         if decode_coords:
    827             var_attrs = new_vars[k].attrs

/Users/Yefee/miniconda2/lib/python2.7/site-packages/xarray/conventions.pyc in decode_cf_variable(var, concat_characters, mask_and_scale, decode_times, decode_endianness)
    764             units = pop_to(attributes, encoding, 'units')
    765             calendar = pop_to(attributes, encoding, 'calendar')
--> 766             data = DecodedCFDatetimeArray(data, units, calendar)
    767         elif attributes['units'] in TIME_UNITS:
    768             # timedelta

/Users/Yefee/miniconda2/lib/python2.7/site-packages/xarray/conventions.pyc in __init__(self, array, units, calendar)
    389             if not PY3:
    390                 msg += ' Full traceback:\n' + traceback.format_exc()
--> 391             raise ValueError(msg)
    392         else:
    393             self._dtype = getattr(result, 'dtype', np.dtype('object'))

ValueError: unable to decode time units u'days since 0000-01-01 00:00:00' with the default calendar. Try opening your dataset with decode_times=False.`

The actual time is:

  31     double time(time) ;

  32         time:long_name = "time" ;

  33         time:units = "days since 0000-01-01 00:00:00" ;

  34         time:bounds = "time_bound" ;

  35         time:calendar = "noleap" ;

I can set ‘decode_times=False’ to open the file but the time is not right.

Coordinates:
    transport_components  (transport_comp) |S256 'Total' ...
    transport_regions     (transport_reg) |S256 'Global Ocean - Marginal Seas' ...
  * time                  (time) float64 396.0```

Is there any suggestions?

[rabernat]

@Yefee I wouldn’t really say this is “solved”. You can bypass the date decoding using decode_times=False, but then you lose lots of useful functionality (e.g. resampling). As described in #521, truly solving the problem will require upstream fixes in numpy and pandas. This is a persistent and highly frustrating problem for the xarray community.

[jhamman]

I don’t think we need another calendar/datetime issue so I’m going to close this. @Yefee, if you’re interested in following along with the progress in this area, see #789.