Handle log colormapping change in bokeh 2.0

Example of the issue as reported by K.-Michael Aye on gitter:

Bokeh 2.x no longer lets the log colormapper handle zeros. I understand the goal of this change was to address this issue https://github.com/bokeh/bokeh/issues/8061 (also see the discussion here https://github.com/bokeh/bokeh/issues/8724). This problem can be more easily avoided when using floats as you can use NaN to avoid the log zero issues.

Unfortunately, this fix will cause a lot of confusion as illustrated by the screenshot above. After discussing this with @jbednar, we listed the following ways we could address this:

1. Document this issue and update how to use clim appropriately, checking and updating all our notebooks and handling all the confusion caused to users who miss the updated docs.

2. Maintain backwards compatibility somehow in bokeh 2.0, stating that our logz semantics match Bokeh > 2.0 and offer a new option for the better behavior (the old behavior resulted in incorrect colorbars). I’m not sure how this could be achieved technically if the old behavior is not available in bokeh 2.x anymore…

3. Somehow support a masking approach at the element level (i.e Image at least, and that could be generalized to other elements?). Seems like a whole lot of work…

4. Address this for the most common use case which will be for any datashading operation that returns zero integers (e.g count). To do this you have to avoid generating integers in the first place as NaN isn’t a valid integer: the idea would be to cast integer arrays to float and replace all zeros with NaNs.

If it isn’t already obvious, I am in favor of option 4 (and maybe option 2, if it is possible) as it would fix the problem in the majority of cases without having to update all the notebooks and inform all affected users. Although I don’t think this is ideal (I would prefer to keep count dtypes as integer) this transformation doesn’t violate datashader semantics where NaN and zero are equivalent. Maybe we could have an option for the operations (default off) to skip this transformation for people who want to preserve integer types?

Unfortunately, I don’t see any easy solutions and nothing that doesn’t involve documenting this change in behavior in Bokeh 2.0 somewhere. What are your thoughts @philippjfr ?

[philippjfr]

I would also vote for throwing a warning in case the clim gets cut off at 1 for integers.

Are you sure this is really needed? What is the alternative here? A zero value can never be displayed in the log color mapper so a lower bound of 1 is the only possible default lower bound for ints.

[philippjfr]

My opinion is that we should warn if the log color mapper encounters a zero value and tell the user to set a clim lower bound. Another option is that if we encounter an integer array we can automatically set the lower bound of the log range to 1 but warn if a float is used since it’s not obvious how to choose a default lower bound.

Somehow support a masking approach at the element level (i.e Image at least, and that could be generalized to other elements?). Seems like a whole lot of work…

We absolutely should do this as it’s needed for masking integer arrays in linked selections. Not sure how quickly we can do this however.

If it isn’t already obvious, I am in favor of option 4 (and maybe option 2, if it is possible) as it would fix the problem in the majority of cases without having to update all the notebooks and inform all affected users.

I’m -1 on both 2 and 4 tbh. There was a reason that log1p was disabled in bokeh because it was effectively wrong in many cases. I’m also not too happy casting ints to floats really.