major_label_overrides dict does not work with numpy scalar keys

This issue is very strange but using a programmatically generated dictionary with xaxis.major_label_overrides fails, whereas using it with a hard-coded dictionary works fine. Even stranger, setting first the hard-coded, and then the programmatically generated also works. Here is an example

Example:

print(sys.version)
print(bokeh.__version__)
x = np.array([1, 2, 3])
y = np.array([5, 3, 8])
a = list("ABC")
d = dict(zip(x, a))
plot = figure(width=300, height=300)
plot.vbar(x=x, top=y, width=1, bottom=0, color=["red", "green", "blue"])
plot.xaxis.ticker = x
assert d == {1: "A", 2: "B", 3: "C"}  # this assert passes
# plot.xaxis.major_label_overrides = {1: "A", 2: "B", 3: "C"} # this works
plot.xaxis.major_label_overrides = d # this doesn't 
show(plot)

Stdout:

3.6.4 |Anaconda, Inc.| (default, Jan 16 2018, 18:10:19) [GCC 7.2.0] 0.13.0

Stacktrace:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-52-6f355b807b3e> in <module>()
     13 # this doesn't
     14 plot.xaxis.major_label_overrides = d
---> 15 show(plot)

~/anaconda3/lib/python3.6/site-packages/bokeh/io/showing.py in show(obj, browser, new, notebook_handle, notebook_url)
    135     if obj not in state.document.roots:
    136         state.document.add_root(obj)
--> 137     return _show_with_state(obj, state, browser, new, notebook_handle=notebook_handle)
    138 
    139 #-----------------------------------------------------------------------------

~/anaconda3/lib/python3.6/site-packages/bokeh/io/showing.py in _show_with_state(obj, state, browser, new, notebook_handle)
    169 
    170     if state.notebook:
--> 171         comms_handle = run_notebook_hook(state.notebook_type, 'doc', obj, state, notebook_handle)
    172         shown = True
    173 

~/anaconda3/lib/python3.6/site-packages/bokeh/io/notebook.py in run_notebook_hook(notebook_type, action, *args, **kw)
    286     if _HOOKS[notebook_type][action] is None:
    287         raise RuntimeError("notebook hook for %r did not install %r action" % notebook_type, action)
--> 288     return _HOOKS[notebook_type][action](*args, **kw)
    289 
    290 #-----------------------------------------------------------------------------

~/anaconda3/lib/python3.6/site-packages/bokeh/io/notebook.py in show_doc(obj, state, notebook_handle)
    491     from ..embed.notebook import notebook_content
    492     comms_target = make_id() if notebook_handle else None
--> 493     (script, div, cell_doc) = notebook_content(obj, comms_target)
    494 
    495     publish_display_data({HTML_MIME_TYPE: div})

~/anaconda3/lib/python3.6/site-packages/bokeh/embed/notebook.py in notebook_content(model, notebook_comms_target, theme)
     80     # has models with the same IDs as they were started with
     81     with _ModelInEmptyDocument(model, apply_theme=theme) as new_doc:
---> 82         (docs_json, [render_item]) = standalone_docs_json_and_render_items([model])
     83 
     84     div = div_for_render_item(render_item)

~/anaconda3/lib/python3.6/site-packages/bokeh/embed/util.py in standalone_docs_json_and_render_items(models)
    338     docs_json = {}
    339     for doc, (docid, _) in docs.items():
--> 340         docs_json[docid] = doc.to_json()
    341 
    342     render_items = []

~/anaconda3/lib/python3.6/site-packages/bokeh/document/document.py in to_json(self)
    779         # this is a total hack to go via a string, needed because
    780         # our BokehJSONEncoder goes straight to a string.
--> 781         doc_json = self.to_json_string()
    782         return loads(doc_json)
    783 

~/anaconda3/lib/python3.6/site-packages/bokeh/document/document.py in to_json_string(self, indent)
    808         }
    809 
--> 810         return serialize_json(json, indent=indent)
    811 
    812     def validate(self):

~/anaconda3/lib/python3.6/site-packages/bokeh/core/json_encoder.py in serialize_json(obj, pretty, indent, **kwargs)
    211         indent = 2
    212 
--> 213     return json.dumps(obj, cls=BokehJSONEncoder, allow_nan=False, indent=indent, separators=separators, sort_keys=True, **kwargs)

~/anaconda3/lib/python3.6/json/__init__.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
    236         check_circular=check_circular, allow_nan=allow_nan, indent=indent,
    237         separators=separators, default=default, sort_keys=sort_keys,
--> 238         **kw).encode(obj)
    239 
    240 

~/anaconda3/lib/python3.6/json/encoder.py in encode(self, o)
    197         # exceptions aren't as detailed.  The list call should be roughly
    198         # equivalent to the PySequence_Fast that ''.join() would do.
--> 199         chunks = self.iterencode(o, _one_shot=True)
    200         if not isinstance(chunks, (list, tuple)):
    201             chunks = list(chunks)

~/anaconda3/lib/python3.6/json/encoder.py in iterencode(self, o, _one_shot)
    255                 self.key_separator, self.item_separator, self.sort_keys,
    256                 self.skipkeys, _one_shot)
--> 257         return _iterencode(o, 0)
    258 
    259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,

TypeError: keys must be a string

[jfx319]

I’m copying over from gitter a related discussion with @bryevdv in case it helps with contributing towards getting to the bottom of this issue.

I also started with numpy when getting tripped up. But later, I removed numpy and observed the following:

major_ticks = [1.5, 2.0, 3.0]
p.xaxis.ticker = FixedTicker(ticks=major_ticks)

major_label_overrides = {1.5: 'foo1', 2.0: 'foo2', 3.0: 'foo3'}

including the “.0” in the the latter two dict_keys prevents the override from happening, while removing the “.0” does activate the override

{1.5: 'foo1', 2: 'foo2', 3: 'foo3'}

This is unexpected, since one might generate the dictionary keys using the original list of ticks values.

The fact that the key=1.5 works, suggests a possible reason: the float values ending in .0’s are likely being stripped of their decimals when converted to string, here? (My .ts knowledge is only from a cursory googling but that seems to be a difference compared to the python side, here, where the key can be either float or str)

workaround:

Convert any whole-number floats to integers when creating the dictionary keys (the tick values themselves need not be converted). I think my typical use case would be: I already know the major_ticks to specify and their corresponding labels --> the dictionary of overrides would be procedurally generated, so that would be the time to do this protection, rather than fixing the tick values themselves.

from bokeh.io import show, output_file, output_notebook
from bokeh.plotting import figure

major_ticks = [1.5, 2.0, 3.0]
major_data_labels = ["foo1", "foo2", "foo3"]

'''
#convert in-place
for index, item in enumerate(major_ticks):
    if float(item).is_integer():
       major_ticks_converted[index] = int(item)
'''
#or, store as a separate list
major_ticks_converted = [int(item)  if float(item).is_integer() else item for item in major_ticks]

p = figure(x_range=(0,4))
p.circle(x=[0.5,0,1,2,2], y=[1,1,1,1,1.5])

p.axis.ticker = major_ticks
major_label_overrides = dict(zip(major_ticks_converted, major_data_labels))
output_notebook()
show(p)

[bryevdv]

Ok, I am inclined to close this again. While the above diff did suppress the error, evidently it did not actually work to display the overrides. The reason is because the accepts always converts numpy scalar to floats (it has to convert them to something), but if the fixed ticks are provided as ints, then they do not get applied. I.e. completely independent of Numpy usage, this works:

plot.xaxis.major_label_overrides = {1: "A", 2: "B", 3: "C"} 

but this does not work:

plot.xaxis.major_label_overrides = {1.0: "A", 2.0: "B", 3.0: "C"} 

Which is presumably because the BokehJS axis code does:

if (ticks[i] in this.model.major_label_overrides)
        labels[i] = this.model.major_label_overrides[ticks[i]]

There’s no way to know ahead of time whether the ticks chosen are or will be floats. So there is no way to know whether numpy scalars would need to be converted to ints or to floats. It’s just a pick-one-and-hope situation. In fact, there is no reason the fixed ticks could not be both, in which case there is no way to pick a single converted type that will work for all the fixed ticks. That makes this "solution:

• fragile
• hard to explain
• involves nontrivial changes to low level code

The only way to make it work fully would be to change the axis JS code above to be some N^2 loop with a further double loop inside that tries out different pairwise type combinations against each other. There is no way this one use case justifies a mess like that, in addition to all the other changes above. Closing with wontfix