Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

bottleneck : Wrong mean for float32 array

See original GitHub issue

I think it is better to have this discussion here instead of on the dask page https://github.com/dask/dask/issues/2095

This is the replicable “bug”:

ds = xarray.open_dataset('/opt/data/ERAIN/ERAIN-t2m-1983-2012.seasmean.nc')
ds.var167.mean()
Out[14]: 
<xarray.DataArray 'var167' ()>
array(261.6441345214844)
ds.var167.data.mean()
Out[15]: 278.62466

The dataset is ~65 MB, here the file https://www.dropbox.com/s/xtj3fm7ihtbwd5r/ERAIN-t2m-1983-2012.seasmean.nc?dl=0 It is a quite normal NetCDF (no NaN), just processed with CDO as you can see on the dask issue.

Issue Analytics

State:
Created 6 years ago
Comments:19 (15 by maintainers)

Top GitHub Comments

3reactions

shoyercommented, Jan 21, 2019

Would it be worth adding a warning (until the right solution is found) if someone is doing .mean() on a DataArray which is float32?

I would rather pick option (1) above, that is, “Stop using bottleneck on float32 arrays”

1reaction

dcheriancommented, May 6, 2022

On second thought we should add this to a FAQ page.

Top Results From Across the Web

bottleneck Documentation

Mean of array elements along given axis ignoring NaNs. float64 intermediate and return values are used for integer inputs. Parameters.

cannot compute moving average with bottleneck package

I'm using package bottleneck in order to compute a moving average of a np array. My data is an array with shape (1247,1)...

Bottleneck distance user manual — gudhi documentation - Inria

Bottleneck distance measures the similarity between two persistence diagrams. It's the shortest distance b for which there exists a perfect matching between ...

Bottleneck - PyPI

import numpy as np >>> a = np.array([1, 2, np.nan, 4, 5]). Find the nanmean: >>> import bottleneck as bn >>> bn.nanmean(a) 3.0....

Storage Latency - Shifting Bottleneck - NetApp Community

Originally SANs were designed with hard disk drives in mind, which meant that the bottleneck was the spinning disks.