geopandas conflict with xarray under python3.7

The following code produce error NetCDF error (OSError: [Errno -101] NetCDF: HDF error: b'data/data.nc') under python3.7.

import geopandas as gpd
import xarray as xr
xr.open_mfdataset('data/*.nc')

The order of importing geopandas is not important. However, following code is working fine.

import xarray as xr
xr.open_mfdataset('data/*.nc')

I guess that some defaults regarding string is being changes by loading geopandas in python3.7. If that is the case, then geopandas may have potential conflict with other libraries too.

version of library used geopandas: 0.5.0 xarray: 0.12.1 python: 3.7.3rc1

[sgillies]

If you install geopandas using pip, without conda, pip will fetch a fiona wheel (binary package) from PyPI. The fiona wheel contains its own copies of libhdf5 and libnetcdf so that users can get started with fiona without having to think about its C library dependencies.

$ unzip -l ~/Downloads/Fiona-1.8.6-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl … 27392688 01-01-1980 00:00 fiona/.dylibs/libgdal.20.dylib 1406264 01-01-1980 00:00 fiona/.dylibs/libgeos-3.6.2.dylib 198572 01-01-1980 00:00 fiona/.dylibs/libgeos_c.1.dylib 6727148 01-01-1980 00:00 fiona/.dylibs/libhdf5.101.dylib 267480 01-01-1980 00:00 fiona/.dylibs/libhdf5_hl.100.dylib 693128 01-01-1980 00:00 fiona/.dylibs/libjpeg.9.dylib 119600 01-01-1980 00:00 fiona/.dylibs/libjson-c.2.dylib 2582008 01-01-1980 00:00 fiona/.dylibs/libnetcdf.11.dylib 655368 01-01-1980 00:00 fiona/.dylibs/libopenjp2.2.3.0.dylib 573628 01-01-1980 00:00 fiona/.dylibs/libpng16.16.dylib 1031508 01-01-1980 00:00 fiona/.dylibs/libproj.12.dylib 2759228 01-01-1980 00:00 fiona/.dylibs/libsqlite3.0.dylib 1244388 01-01-1980 00:00 fiona/.dylibs/libtiff.5.dylib

When you import geopandas, the hdf5 and netcdf libraries are loaded (via fiona).

xarray depends on the netCDF4 package to read and write files and that package also includes libnetcdf and libhdf5 libraries.

$ unzip -l ~/Downloads/netCDF4-1.5.1.2-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl … 77024 01-01-1980 00:00 netCDF4/.dylibs/libaec.0.dylib 704468 01-01-1980 00:00 netCDF4/.dylibs/libcurl.4.dylib 13604560 01-01-1980 00:00 netCDF4/.dylibs/libhdf5.101.dylib 366680 01-01-1980 00:00 netCDF4/.dylibs/libhdf5_hl.100.dylib 3168800 01-01-1980 00:00 netCDF4/.dylibs/libnetcdf.15.dylib 33876 01-01-1980 00:00 netCDF4/.dylibs/libsz.2.dylib

My understanding of the problem is this: when xarray imports netCDF4 (in open_mfdataset), your system’s loader finds that libnetcdf has already been loaded (by fiona) and moves on. The problem is that the loaded library is the wrong version. The Fiona and netCDF4 wheels are packaged using the same approach, but due to lack of coordination between the packagers, they are incompatible.

With conda-forge, there is a single packager, and so everything can be made compatible in theory. In practice, conda-forge can be broken for short periods of time when new versions of packages are added and compatibility is worked out.

[adrHuerta]

I have the same issue like @rafa-guedes, but in python3.6x