question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

open_dataset crashes for coordinate grids with a single node

See original GitHub issue

Hello cfgrib gurus,

Experienced python user here, but complete novice to cfgrib and the GRIB format itself. I am trying to look at an ERA5 GRIB file in which the coordinate grid contains a unique latitude/longitude entry (i.e. looking at the ERA5 data for a single point on Earth).

Calling xr.open_dataset() gets me the following error:

~/anaconda3/lib/python3.7/site-packages/cfgrib/dataset.py in build_geography_coordinates(index, encode_cf, errors, log)
    366             dimensions=('latitude',), data=latitudes, attributes=COORD_ATTRS['latitude'].copy()
    367         )
--> 368         if latitudes[0] > latitudes[-1]:
    369             geo_coord_vars['latitude'].attributes['stored_direction'] = 'decreasing'
    370         geo_coord_vars['longitude'] = Variable(

IndexError: too many indices for array

I went in cfgrib/dataset.py to see if I might bypass this using an easy-but-far-from-perfect fix, doing:

if latitudes.ndim > 0: # <= this just by-passes the check
    if latitudes[0] > latitudes[-1]:
        geo_coord_vars['latitude'].attributes['stored_direction'] = 'decreasing'

However, this only displaces the problem further down the line, as I am then faced with the following error:

[...]
~/anaconda3/lib/python3.7/site-packages/xarray/backends/cfgrib_.py in open_store_variable(self, name, var)
     53         encoding["original_shape"] = var.data.shape
     54 
---> 55         return Variable(var.dimensions, data, var.attributes, encoding)
     56 
     57     def get_variables(self):

~/anaconda3/lib/python3.7/site-packages/xarray/core/variable.py in __init__(self, dims, data, attrs, encoding, fastpath)
    302         """
    303         self._data = as_compatible_data(data, fastpath=fastpath)
--> 304         self._dims = self._parse_dimensions(dims)
    305         self._attrs = None
    306         self._encoding = None

~/anaconda3/lib/python3.7/site-packages/xarray/core/variable.py in _parse_dimensions(self, dims)
    498             raise ValueError(
    499                 "dimensions %s must have the same length as the "
--> 500                 "number of data dimensions, ndim=%s" % (dims, self.ndim)
    501             )
    502         return dims

ValueError: dimensions ('latitude',) must have the same length as the number of data dimensions, ndim=0

To me, it looks like the “real” issue is that for coordinate grids with single nodes, the latitudes and longitudes values are stored in 0-dimensional arrays ?

Running Mac OS 10.13.6 with Python 3.7.3, cfgrib 0.9.8.1, and xarray 0.15.1 and pyeccodes (in place of the ecCodes library). For completeness:

$ python -m cfgrib selfcheck
Found: ecCodes v2.17.0.
Your system is ready.

Here is a demo ERA5 GRIB file that allows to reproduce the problem. ERA5_single-grid-point_test.grib.zip

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
fpavogtcommented, Apr 9, 2021

@alexamici: great ! Thanks a lot for the fix @iainrussell !

1reaction
iainrussellcommented, Apr 8, 2021

Looks like this is what I fixed for #199.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Common errors that may occur during editing—Help
Errors that may occur when you start editing · Projected coordinate systems · Coordinates or measures are out of bounds · The spatial...
Read more >
Datasets
Datasets. This module provides benchmark datasets for graph-level and node-level prediction. Datasets are automatically downloaded and saved locally on ...
Read more >
Best practices to go from 1000s of netcdf files to analyses ...
Advice to make these HYCOM data analysis-ready for pangeo tools, favoring the time dimension/coordinates. Conducting some tests and visualizing ...
Read more >
UCI Machine Learning Repository: Data Sets
Name Data Types Default Task Attribute Types # Instances # Attributes Ye... Abalone Multivariate Classification Categorical, Integer, Real 4177 8 19... Adult Multivariate Classification Categorical,...
Read more >
How safe are the streets of Santiago? | by Marcelo Rovai
A join between the above fileshape and the crashes dataset. The resultant dataset will have for each line a point (crash) and a...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found