Trying to display heat map of dataset

I am trying to draw a heat map of a dataset on a map following the example on the NYC taxi.

My data read looks as follows

data_sources = pd.read_csv("https://s3.amazonaws.com/fordgobike-data/2017-fordgobike-tripdata.csv")

Next I seek to aggregate and plot

url = 'https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{Z}/{Y}/{X}.jpg'
tile_opts  = dict(width=1000,height=600,xaxis=None,yaxis=None,bgcolor='black',show_grid=False)
map_tiles  = gv.WMTS(url).opts(style=dict(alpha=0.5), plot=tile_opts)
points     = hv.Points(df, ['start_station_longitude', 'start_station_latitude'])
bike_trips = datashade(points, x_sampling=1, y_sampling=1, cmap=fire, width=1000, height=600)

points * bike_trips

this gives me the following error

WARNING:root:dynamic_operation: Exception raised in callable 'dynamic_operation' of type 'function'.
Invoked as dynamic_operation(height=600, scale=1.0, width=1000, x_range=None, y_range=None)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/IPython/core/formatters.py in __call__(self, obj, include, exclude)
    968 
    969             if method is not None:
--> 970                 return method(include=include, exclude=exclude)
    971             return None
    972         else:

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/dimension.py in _repr_mimebundle_(self, include, exclude)
   1229         combined and returned.
   1230         """
-> 1231         return Store.render(self)
   1232 
   1233 

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/options.py in render(cls, obj)
   1287         data, metadata = {}, {}
   1288         for hook in hooks:
-> 1289             ret = hook(obj)
   1290             if ret is None:
   1291                 continue

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/ipython/display_hooks.py in pprint_display(obj)
    278     if not ip.display_formatter.formatters['text/plain'].pprint:
    279         return None
--> 280     return display(obj, raw_output=True)
    281 
    282 

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/ipython/display_hooks.py in display(obj, raw_output, **kwargs)
    254     elif isinstance(obj, (HoloMap, DynamicMap)):
    255         with option_state(obj):
--> 256             output = map_display(obj)
    257     elif isinstance(obj, Plot):
    258         output = render(obj)

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/ipython/display_hooks.py in wrapped(element)
    140         try:
    141             max_frames = OutputSettings.options['max_frames']
--> 142             mimebundle = fn(element, max_frames=max_frames)
    143             if mimebundle is None:
    144                 return {}, {}

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/ipython/display_hooks.py in map_display(vmap, max_frames)
    208         return None
    209 
--> 210     return render(vmap)
    211 
    212 

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/ipython/display_hooks.py in render(obj, **kwargs)
     63         renderer = renderer.instance(fig='png')
     64 
---> 65     return renderer.components(obj, **kwargs)
     66 
     67 

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/plotting/bokeh/renderer.py in components(self, obj, fmt, comm, **kwargs)
    257         # Bokeh has to handle comms directly in <0.12.15
    258         comm = False if bokeh_version < '0.12.15' else comm
--> 259         return super(BokehRenderer, self).components(obj,fmt, comm, **kwargs)
    260 
    261 

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/plotting/renderer.py in components(self, obj, fmt, comm, **kwargs)
    319             plot = obj
    320         else:
--> 321             plot, fmt = self._validate(obj, fmt)
    322 
    323         widget_id = None

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/plotting/renderer.py in _validate(self, obj, fmt, **kwargs)
    218         if isinstance(obj, tuple(self.widgets.values())):
    219             return obj, 'html'
--> 220         plot = self.get_plot(obj, renderer=self, **kwargs)
    221 
    222         fig_formats = self.mode_formats['fig'][self.mode]

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/plotting/bokeh/renderer.py in get_plot(self_or_cls, obj, doc, renderer)
    150             doc = Document() if self_or_cls.notebook_context else curdoc()
    151         doc.theme = self_or_cls.theme
--> 152         plot = super(BokehRenderer, self_or_cls).get_plot(obj, renderer)
    153         plot.document = doc
    154         return plot

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/plotting/renderer.py in get_plot(self_or_cls, obj, renderer)
    185         """
    186         # Initialize DynamicMaps with first data item
--> 187         initialize_dynamic(obj)
    188 
    189         if not isinstance(obj, Plot):

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/plotting/util.py in initialize_dynamic(obj)
    242             continue
    243         if not len(dmap):
--> 244             dmap[dmap._initial_key()]
    245 
    246 

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/spaces.py in __getitem__(self, key)
   1123         # Not a cross product and nothing cached so compute element.
   1124         if cache is not None: return cache
-> 1125         val = self._execute_callback(*tuple_key)
   1126         if data_slice:
   1127             val = self._dataslice(val, data_slice)

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/spaces.py in _execute_callback(self, *args)
    909 
    910         with dynamicmap_memoization(self.callback, self.streams):
--> 911             retval = self.callback(*args, **kwargs)
    912         return self._style(retval)
    913 

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/spaces.py in __call__(self, *args, **kwargs)
    540         kwarg_hash = kwargs.pop('memoization_hash', ())
    541         (self.args, self.kwargs) = (args, kwargs)
--> 542         if not args and not kwargs: return self.callable()
    543         inputs = [i for i in self.inputs if isinstance(i, DynamicMap)]
    544         streams = []

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/overlay.py in dynamic_mul(*args, **kwargs)
     27             from .spaces import Callable
     28             def dynamic_mul(*args, **kwargs):
---> 29                 element = other[args]
     30                 return self * element
     31             callback = Callable(dynamic_mul, inputs=[self, other])

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/spaces.py in __getitem__(self, key)
   1123         # Not a cross product and nothing cached so compute element.
   1124         if cache is not None: return cache
-> 1125         val = self._execute_callback(*tuple_key)
   1126         if data_slice:
   1127             val = self._dataslice(val, data_slice)

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/spaces.py in _execute_callback(self, *args)
    909 
    910         with dynamicmap_memoization(self.callback, self.streams):
--> 911             retval = self.callback(*args, **kwargs)
    912         return self._style(retval)
    913 

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/spaces.py in __call__(self, *args, **kwargs)
    570 
    571         try:
--> 572             ret = self.callable(*args, **kwargs)
    573         except KeyError:
    574             # KeyError is caught separately because it is used to signal

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/util/__init__.py in dynamic_operation(*key, **kwargs)
    431                 self.p.kwargs.update(kwargs)
    432                 obj = map_obj[key] if isinstance(map_obj, HoloMap) else map_obj
--> 433                 return self._process(obj, key)
    434         else:
    435             def dynamic_operation(*key, **kwargs):

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/util/__init__.py in _process(self, element, key)
    417             kwargs = {k: v for k, v in self.p.kwargs.items()
    418                       if k in self.p.operation.params()}
--> 419             return self.p.operation.process_element(element, key, **kwargs)
    420         else:
    421             return self.p.operation(element, **self.p.kwargs)

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/operation.py in process_element(self, element, key, **params)
    141         """
    142         self.p = param.ParamOverrides(self, params)
--> 143         return self._apply(element, key)
    144 
    145 

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/operation.py in _apply(self, element, key)
    119         for hook in self._preprocess_hooks:
    120             kwargs.update(hook(self, element))
--> 121         ret = self._process(element, key)
    122         for hook in self._postprocess_hooks:
    123             ret = hook(self, ret, **kwargs)

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/operation/datashader.py in _process(self, element, key)
    911 
    912     def _process(self, element, key=None):
--> 913         agg = rasterize._process(self, element, key)
    914         shaded = shade._process(self, agg, key)
    915         return shaded

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/operation/datashader.py in _process(self, element, key)
    742             op = transform.instance(**op_params)
    743             op._precomputed = self._precomputed
--> 744             element = element.map(op, predicate)
    745             self._precomputed = op._precomputed
    746         return element

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/dimension.py in map(self, map_fn, specs, clone)
    692             return deep_mapped
    693         else:
--> 694             return map_fn(self) if applies else self
    695 
    696 

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/operation.py in __call__(self, element, **params)
    161                                 operation=self, kwargs=params)
    162         elif isinstance(element, ViewableElement):
--> 163             processed = self._apply(element)
    164         elif isinstance(element, DynamicMap):
    165             if any((not d.values) for d in element.kdims):

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/operation.py in _apply(self, element, key)
    119         for hook in self._preprocess_hooks:
    120             kwargs.update(hook(self, element))
--> 121         ret = self._process(element, key)
    122         for hook in self._postprocess_hooks:
    123             ret = hook(self, ret, **kwargs)

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/operation/datashader.py in _process(self, element, key)
    460             # Replacing x and y coordinates to avoid numerical precision issues
    461             eldata = agg if ds_version > '0.5.0' else (xs, ys, agg.data)
--> 462             return self.p.element_type(eldata, **params)
    463         else:
    464             layers = {}

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/element/raster.py in __init__(self, data, kdims, vdims, bounds, extents, xdensity, ydensity, rtol, **params)
    280         xdensity = xdensity if xdensity else compute_density(l, r, dim1, self._time_unit)
    281         ydensity = ydensity if ydensity else compute_density(b, t, dim2, self._time_unit)
--> 282         SheetCoordinateSystem.__init__(self, bounds, xdensity, ydensity)
    283         self._validate(data_bounds, supplied_bounds)
    284 

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/sheetcoords.py in __init__(self, bounds, xdensity, ydensity)
    164         self.lbrt = np.array(bounds.lbrt())
    165 
--> 166         r1,r2,c1,c2 = Slice._boundsspec2slicespec(self.lbrt,self)
    167         self.__shape = (r2-r1,c2-c1)
    168 

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/sheetcoords.py in _boundsspec2slicespec(boundsspec, scs)
    516         b_m,r_m = scs.sheet2matrix(r,b)
    517 
--> 518         l_idx = int(np.ceil(l_m-0.5))
    519         t_idx = int(np.ceil(t_m-0.5))
    520         # CBENHANCEMENT: Python 2.6's math.trunc()?

ValueError: cannot convert float NaN to integer

For what’s worth, trying to plot the dots on a map using

(gv.WMTS('http://c.tile.openstreetmap.org/{Z}/{X}/{Y}.png') * points)

gives the following plot

My environment.yml looks as follows

name: example
channels:
 - bokeh
 - conda-forge
 - ioam

dependencies:
 - attrs
 - beautifulsoup4
 - bokeh
 - cartopy
 - colorcet
 - conda-forge::graphviz
 - conda-forge::pytest
 - conda-forge::pytest-benchmark
 - conda-forge::python-graphviz
 - dask>=0.15.4
 - datashader
 - dill
 - distributed
 - fastparquet
 - flake8
 - geoviews
 - ioam::holoviews>=1.8.3
 - ipython
 - iris
 - jupyter
 - jupyter_dashboards
 - krb5
 - matplotlib
 - nbconvert
 - nbformat
 - networkx>=2.0
 - numba
 - numpy
 - pandas
 - param>=1.5.1
 - paramnb
 - pyproj
 - pytables
 - python-snappy
 - python=3.6
 - rasterio
 - requests
 - scikit-image
 - scipy
 - shapely
 - snappy
 - statsmodels
 - tblib
 - xarray
 - yaml
 - xgboost=0.71
 - autopep8
 - jupyter_contrib_nbextensions
 - pip:
   - cachey
   - streamz==0.2.0
   - webargs

Any suggestions how I could generate the heat map?

[jbednar]

It works if you remove x_sampling=1 and y_sampling=1. A value of 1 was appropriate for the Web Mercator Taxi data (where the 1 is in meters), but for this lon/lat data not previously projected into Web Mercator, “1” is one degree on the globe. All of the data appears to fall into one degree on the globe, so enforcing that as a minimum sampling size effectively gives a single-pixel plot, which gives no valid data and ends up with that obscure and impressively long traceback above. I think we already have an issue or PR to make that case not be an error, but for the meantime just fix those settings.

You also presumably didn’t mean to overlay bike_trips on points, as they are both the same set of points; I assume you meant map_tiles. The result:

import holoviews as hv, geoviews as gv, pandas as pd
from holoviews.operation.datashader import datashade
from colorcet import fire
hv.extension("bokeh")

df = pd.read_csv("https://s3.amazonaws.com/fordgobike-data/2017-fordgobike-tripdata.csv")
url = 'https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{Z}/{Y}/{X}.jpg'
tile_opts  = dict(width=1000,height=600,xaxis=None,yaxis=None,bgcolor='black',show_grid=False, alpha=0.3)
map_tiles  = gv.WMTS(url).options(**tile_opts)
points     = gv.Points(df, ['start_station_longitude', 'start_station_latitude'])
bike_trips = datashade(points, cmap=fire, width=1000, height=600)

map_tiles * bike_trips

Maybe not the most exciting Datashader plot ever, but it seems to work…

[philippjfr]

Right, though I think @philippjfr and I agreed earlier this shouldn’t be an error at the HoloViews level, and probably not at the Datashader level either. It should just render the same value across the entire image.

Correct, I think this can probably be handled but separately a single pixel Image should provide a good error message when when xdensity and ydensity aren’t supplied.