variable name error in pydap 3.2.2

I’m having trouble accessing a dataset with pydap 3.2.2 that worked fine in 3.2.1. I think 3.2.2 mangles the variable name. I briefly looked at the source, but it wasn’t clear to me where the problem was.

pydap 3.2.1

>>> from pydap.client import open_url
>>> dataset = open_url('https://nomads.ncdc.noaa.gov/thredds/dodsC/gfs-004/201612/20161201/gfs_4_20161201_0000_003.grb2')
>>> tsurf = dataset['Temperature_surface']
>>> tsurf[0, 0, 0]
<GridType with array 'Temperature_surface' and maps 'time', 'lat', 'lon'>

pydap 3.2.2

>>> from pydap.client import open_url
>>> dataset = open_url('https://nomads.ncdc.noaa.gov/thredds/dodsC/gfs-004/201612/20161201/gfs_4_20161201_0000_003.grb2')
>>> tsurf = dataset['Temperature_surface']
>>> tsurf[0, 0, 0]
Traceback (most recent call last):
  File "/Users/holmgren/miniconda3/envs/pydap322/lib/python3.6/site-packages/pydap/model.py", line 404, in _getitem_string
    return self._dict[quote(key)]
KeyError: 'Temperature_surface%2ETemperature_surface'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/holmgren/miniconda3/envs/pydap322/lib/python3.6/site-packages/pydap/model.py", line 740, in __getitem__
    var.data = self[var.name].data[slice_]
  File "/Users/holmgren/miniconda3/envs/pydap322/lib/python3.6/site-packages/pydap/handlers/dap.py", line 149, in __getitem__
    return dataset[self.id].data
  File "/Users/holmgren/miniconda3/envs/pydap322/lib/python3.6/site-packages/pydap/model.py", line 425, in __getitem__
    return self._getitem_string(key)
  File "/Users/holmgren/miniconda3/envs/pydap322/lib/python3.6/site-packages/pydap/model.py", line 409, in _getitem_string
    return self[splitted[0]]['.'.join(splitted[1:])]
  File "/Users/holmgren/miniconda3/envs/pydap322/lib/python3.6/site-packages/pydap/model.py", line 320, in __getitem__
    out.data = self._get_data_index(index)
  File "/Users/holmgren/miniconda3/envs/pydap322/lib/python3.6/site-packages/pydap/model.py", line 349, in _get_data_index
    return self._data[index]
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices

[jalbert10NJ]

Apparently the above issue is due to the fact that one needs to fully pip uninstall Pydap before doing a pip install from a git repository, e.g. python -m pip install git+https://github.com/pydap/pydap.git@fc0dd0e6f180e455dbb68a06a85f84b1eca6754f (or just pip install git+https://github.com/pydap/pydap.git@fc0dd0e6f180e455dbb68a06a85f84b1eca6754f ), since if one already has Pydap 3.2.2 installed, then the pip install git+https://github.com/pydap/pydap.git@fc0dd0e6f180e455dbb68a06a85f84b1eca6754f does not install the update (likely due to a completely separate bug in pip – at least in the pip in the anaconda install of python 3.7 on MacOS Mojave – and despite the fact that the pip install does properly create the wheel, it does not properly install it if Pydap 3.2.2 has already been installed). (The wheel just rolls away 😉

Anyway, IMO there is no reason that this bug should still remain in the default version of Pydap, and require people to pip install from github, rather than from the default PyPI. If Pydap maintainers are waiting for other fixes before releasing 3.2.3, as is apparently the case, then IMO a 3.2.2.1 with this fix should be released to PyPI in the meantime. Additional people should not have to waste weeks re-debugging this issue. I am thus opening this issue back up (and request that it not be closed until the default version of Pydap from PyPI contains this fix).

[ghost]

I’m having similar trouble when using xarray and pydap. Why the variable name (key) gets doubled like above: ‘Temperature_surface.Temperature_surface’?