Getting TypeError: Object of type Polygon is not JSON serializable after show()

Hello everyone,

First time posting on the form here. It was really nice to e-meeting you guys. I have been struggled with this issue for couple days and need some help.

I am trying to use CustomJS to callback some filter data after select dropdown list and graph out selected the geometry on the top of an US map.

The logic is: using Geopandas to read the shape file -> convert it to dictionary to fit into ColumnDataSource -> using CustomJS to recall the filtered data I apply -> convert it back Json by using to_json() -> apply Geopandas dataframe -> graph using GeoJSONDataSource.

This is the code:

# reset the graph
reset_output()

# import data
data = data[~data.STUSPS.isin(['AK','AS', 'GU', 'HI', 'PR','MP', 'VI'])]
data3 = data[data.STUSPS.isin(['TX', 'UT'])]
data3_m = data3.drop(['geometry'], axis=1)

# create filter
states = sorted(list(data2.NAME.unique()))
select = Select(title="State", options=states)

# Get source
source = ColumnDataSource(ColumnDataSource.from_df(data3_m.loc[:]))
source2 = ColumnDataSource(ColumnDataSource.from_df(data3.loc[:]))

# create filtered source
filteredSource = ColumnDataSource(data=dict(STATEFP=[],NAME=[],ALAND=[]))
geo_filteredSource = GeoJSONDataSource(geojson=gpd.GeoDataFrame(dict(STATEFP=[],NAME=[],ALAND=[],geometry=[])).to_json())

# start column table
columns = [TableColumn(field="STATEFP",title="STATEFP",sortable=True),
           TableColumn(field="NAME",title="NAME",sortable=True),
           TableColumn(field="ALAND",title="ALAND",sortable=True),
           TableColumn(field="geometry",title="geometry",sortable=True)]

data_table=DataTable(source=filteredSource,columns=columns, width=800)

# <----- Call back ----->
callback = CustomJS(args=dict(source=source,
                              filteredSource=filteredSource,
                              data_table=data_table), code="""
var data = source.data;
var f = cb_obj.value;
var d2 = filteredSource.data;
d2['NAME']=[]
d2['STATEFP']=[]
d2['ALAND']=[]


for(i = 0; i < data['NAME'].length;i++){

if(data['NAME'][i]==f){

    d2['NAME'].push(data['NAME'][i])
    d2['STATEFP'].push(data['STATEFP'][i])
    d2['ALAND'].push(data['ALAND'][i])
}

}

filteredSource.change.emit()
// trigger change on datatable
data_table.change.emit()

""")

callback2 = CustomJS(args=dict(source=source2,
                              filteredSource=geo_filteredSource), code="""
var data = source.data;
var f = cb_obj.value;
var d2 = filteredSource.data;
d2['NAME']=[]
d2['STATEFP']=[]
d2['ALAND']=[]
d2['geometry']=[]


for(i = 0; i < data['NAME'].length;i++){

if(data['NAME'][i]==f){

    d2['NAME'].push(data['NAME'][i])
    d2['STATEFP'].push(data['STATEFP'][i])
    d2['ALAND'].push(data['ALAND'][i])
    d2['geometry'].push(data['geometry'][i])
}

}

filteredSource.change.emit()
""")

select.js_on_change('value', callback)
select.js_on_change('value', callback2)

# <----- Draw Graph ---->
# Export to geojson
geojson = data1.to_json()

# get column name for filter
color_column = 'ALAND'

# transform to geojson file
geo_source = GeoJSONDataSource(geojson=geojson)

# create gradient color
color_mapper = LinearColorMapper(palette='Viridis256', low=min(data1[color_column]), high=max(data1[color_column]))

# start the figure
p = figure(
    title="USA", 
    x_axis_location=None, y_axis_location=None,
    plot_width=1000, plot_height=600)

p.patches('xs', 'ys', fill_alpha=1,
          fill_color = '#ffffff',
          line_color='black', line_width=0.5, source=geo_source)

r2 = p.patches('xs', 'ys', fill_alpha=0.7,
          fill_color={'field': color_column, 'transform': color_mapper},
          line_color='black', line_width=0.5, source=geo_filteredSource)

p.add_tools(HoverTool(renderers=[r2], tooltips=[("Name", "@NAME"), ("ALAND", "@ALAND")]))

# Add color bar to the right
color_bar = ColorBar(color_mapper=color_mapper, ticker=LogTicker(),
                     label_standoff=12, border_line_color=None, location=(0,0))
p.add_layout(color_bar, 'right')

layout = column(widgetbox(select), p, widetbox(data_table))

show(layout)

everything looks okay at the time, however, once I applied show(layout). it is giving me this TypeError: Object of type Polygon is not JSON serializable. Any idea on how I can fix it?

Thank you so much and have a nice day.

Sincerely,

David

Below is the error code:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-490-54a49b4db53d> in <module>()
    141 layout = column(widgetbox(select, data_table))
    142 
--> 143 show(layout)

/anaconda3/lib/python3.7/site-packages/bokeh/io/showing.py in show(obj, browser, new, notebook_handle, notebook_url)
    135     if obj not in state.document.roots:
    136         state.document.add_root(obj)
--> 137     return _show_with_state(obj, state, browser, new, notebook_handle=notebook_handle)
    138 
    139 #-----------------------------------------------------------------------------

/anaconda3/lib/python3.7/site-packages/bokeh/io/showing.py in _show_with_state(obj, state, browser, new, notebook_handle)
    173 
    174     if state.file or not shown:
--> 175         _show_file_with_state(obj, state, new, controller)
    176 
    177     return comms_handle

/anaconda3/lib/python3.7/site-packages/bokeh/io/showing.py in _show_file_with_state(obj, state, new, controller)
    156 
    157     '''
--> 158     filename = save(obj, state=state)
    159     controller.open("file://" + filename, new=NEW_PARAM[new])
    160 

/anaconda3/lib/python3.7/site-packages/bokeh/io/saving.py in save(obj, filename, resources, title, template, state, **kwargs)
     81 
     82     filename, resources, title = _get_save_args(state, filename, resources, title)
---> 83     _save_helper(obj, filename, resources, title, template)
     84     return abspath(filename)
     85 

/anaconda3/lib/python3.7/site-packages/bokeh/io/saving.py in _save_helper(obj, filename, resources, title, template)
    143     '''
    144     from ..embed import file_html
--> 145     html = file_html(obj, resources, title=title, template=template)
    146 
    147     with io.open(filename, mode="w", encoding="utf-8") as f:

/anaconda3/lib/python3.7/site-packages/bokeh/embed/standalone.py in file_html(models, resources, title, template, template_variables, theme)
    279 
    280     with _ModelInDocument(models, apply_theme=theme) as doc:
--> 281         (docs_json, render_items) = standalone_docs_json_and_render_items(models)
    282         title = _title_from_models(models, title)
    283         bundle = bundle_for_objs_and_resources([doc], resources)

/anaconda3/lib/python3.7/site-packages/bokeh/embed/util.py in standalone_docs_json_and_render_items(models)
    338     docs_json = {}
    339     for doc, (docid, _) in docs.items():
--> 340         docs_json[docid] = doc.to_json()
    341 
    342     render_items = []

/anaconda3/lib/python3.7/site-packages/bokeh/document/document.py in to_json(self)
    779         # this is a total hack to go via a string, needed because
    780         # our BokehJSONEncoder goes straight to a string.
--> 781         doc_json = self.to_json_string()
    782         return loads(doc_json)
    783 

/anaconda3/lib/python3.7/site-packages/bokeh/document/document.py in to_json_string(self, indent)
    808         }
    809 
--> 810         return serialize_json(json, indent=indent)
    811 
    812     def validate(self):

/anaconda3/lib/python3.7/site-packages/bokeh/core/json_encoder.py in serialize_json(obj, pretty, indent, **kwargs)
    211         indent = 2
    212 
--> 213     return json.dumps(obj, cls=BokehJSONEncoder, allow_nan=False, indent=indent, separators=separators, sort_keys=True, **kwargs)

/anaconda3/lib/python3.7/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
    236         check_circular=check_circular, allow_nan=allow_nan, indent=indent,
    237         separators=separators, default=default, sort_keys=sort_keys,
--> 238         **kw).encode(obj)
    239 
    240 

/anaconda3/lib/python3.7/json/encoder.py in encode(self, o)
    197         # exceptions aren't as detailed.  The list call should be roughly
    198         # equivalent to the PySequence_Fast that ''.join() would do.
--> 199         chunks = self.iterencode(o, _one_shot=True)
    200         if not isinstance(chunks, (list, tuple)):
    201             chunks = list(chunks)

/anaconda3/lib/python3.7/json/encoder.py in iterencode(self, o, _one_shot)
    255                 self.key_separator, self.item_separator, self.sort_keys,
    256                 self.skipkeys, _one_shot)
--> 257         return _iterencode(o, 0)
    258 
    259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,

/anaconda3/lib/python3.7/site-packages/bokeh/core/json_encoder.py in default(self, obj)
    133 
    134         else:
--> 135             return self.transform_python_types(obj)
    136 
    137 def serialize_json(obj, pretty=None, indent=None, **kwargs):

/anaconda3/lib/python3.7/site-packages/bokeh/core/json_encoder.py in transform_python_types(self, obj)
    100 
    101         else:
--> 102             return super(BokehJSONEncoder, self).default(obj)
    103 
    104     def default(self, obj):

/anaconda3/lib/python3.7/json/encoder.py in default(self, o)
    177 
    178         """
--> 179         raise TypeError(f'Object of type {o.__class__.__name__} '
    180                         f'is not JSON serializable')
    181 

TypeError: Object of type Polygon is not JSON serializable

[troyswanson]

For anyone from the internet who might stumble across this issue, if your source data contains a column of geometry objects from a library such as shapely, Bokeh will not know how to package the data to JSON so that it can send it to the JavaScript rendering system.

The solution that I used was to drop the column containing the polygon data from the dataframe when creating the ColumnDataSource object, like so:

cds = ColumnDataSource(df.drop(columns=['geometry']))

Then, when creating your figure, you can include this as the source without issue.

Hope this helps!

[troyswanson]

After reading my original response, it’s not as complete of an explanation as it could be.

Bokeh has two parts – Python and JavaScript. The Python part is used in a Jupyter Notebook or a Python script (or really anywhere you use Python). The JavaScript part is used to render the figure in the web browser.

With that in mind, there has to be a way for data to be communicated from Python to JavaScript. One way is to include the data directly in the Python method call. The method will package the data and pass it to Javascript for rendering.

Another way is through a ColumnDataSource object, which is very convenient if you are working with a data frame object from Pandas. The ColumnDataSource object takes all of the data in the data frame and tries to convert it to JSON so that it can be passed to the JavaScript part of Bokeh for rendering. The problem is that if you have objects in your data frame that cannot be serialized to JSON (eg: shapely objects), then the JavaScript part of Bokeh won’t be able to read the data passed from the Python part.

The way to solve this is to extract the data that Bokeh needs from the shapely object and store that in columns that can be serialized to JSON, and then drop the column containing the shapely objects.

Here’s some code that demonstrates this in a Jupyter Notebook:

from shapely.geometry import Polygon

import pandas as pd

from bokeh.plotting import figure, output_notebook, show
from bokeh.models import ColumnDataSource

output_notebook()

# create a shapely polygon object
geom = Polygon([(3, 1), (5, 3.5), (4, 5), (3, 4), (2, 5), (1, 3.5), (3, 1)])

# create a pandas dataframe with a geometry column containing the shapely object
df = pd.DataFrame([geom], columns=['geometry'])

# extract the x/y data from the shapely object
df['x'] = df['geometry'].apply(lambda g: list(g.boundary.xy[0]))
df['y'] = df['geometry'].apply(lambda g: list(g.boundary.xy[1]))

# create the ColumnDataSource object for bokeh
# remember to drop the column containing the shapely objects
cds = ColumnDataSource(df.drop(columns=['geometry']))

# draw the bokeh figure
p = figure(height=500, width=500)
p.patches('x', 'y', color='red', fill_alpha=0.5, source=cds)
show(p)

At the end of this script, you should see something like this: