nbytes flicker prevention causes high scheduler CPU

Unfortunately, I think the slowdown we were concerned about with https://github.com/dask/distributed/pull/4997#discussion_r661831758 turns out to be significant. Scheduler CPU, when totally idle with no workers, is now around 20%

1. Run dask-scheduler
2. Go directly to the system plot (http://localhost:8787/system)
3. Now click on the status plot (http://localhost:8787/status)
4. Now go back to the system plot; CPU is much higher

Before #4997 (c6a12d843f91efcefd3bd020ed0eb5165a1ab808) #4997 (8b79bd95d0c5a681f6d0543eb20d645dc1182ec1)

Here’s a py-spy profile: https://www.speedscope.app/#profileURL=https%3A%2F%2Fgistcdn.githack.com%2Fgjoseph92%2F689bfa3543a47708d0b0e3d1e236b150%2Fraw%2Fad939f0cb54d5f592a110fa34aa22b51e13b9a9e%2Fidle-main.json. Make sure to go to MainThread with the thread selector, and check out the left heavy view. We’re spending most of the time on https://github.com/dask/distributed/blob/8b79bd95d0c5a681f6d0543eb20d645dc1182ec1/distributed/dashboard/components/scheduler.py#L509 and https://github.com/dask/distributed/blob/8b79bd95d0c5a681f6d0543eb20d645dc1182ec1/distributed/dashboard/components/scheduler.py#L345 constructing the DataRange1d.

FWIW I did a git bisect to be extra sure that it was actually this change, and indeed it was 😦

cc @jrbourbeau @ncclementi

[bryevdv]

If you mean to use dumb (non-autoranging) ranges, then this

self.root.x_range = range(0, x_end)

Should be

self.root.x_range = Range1d(0, x_end)

The range function is Python’s builtin range generator, not one of Bokeh’s models.

Otherwise, offhand I would say _invalidate_all_models will occur when models in the document are added, deleted, etc. It’s not really possible to speculate without knowing what func does. The general guidance to set up as much as possible up front and make the smallest changes is advised, e.g. as an example: if the choice is between adding every glyph up front and toggling their visibility as needed later (cheap, lightweight), or actually deleting and adding entire glyph objects later (expensive, heavyweight), choose the first option if possible.

[jakirkham]

cc @bryevdv (in case you have suggestions here)