When viewer.add_image() is called on a very large float array without specifying contrast_limits, the data range calculation overwhelms system resources.

🐛 Bug

When viewer.add_image() is called on a very large float array without specifying contrast_limits, the data range calculation overwhelms system resources.

To Reproduce

Steps to reproduce the behavior:

The following script lazily generates a very large array. Wrapper function can be used to count the number of array-generating calls.

import dask.array as da
from dask import delayed
import numpy as np
import napari
import itertools

def counter(func, counter_dict={}):
    counter_dict[func]=0
    def wrap(*args,**kwargs):
        counter_dict[func] += 1
        print("Number of times {} has been called so far {}".format(func.__name__, counter_dict[func]))
        return func(*args,**kwargs)
    return wrap

@counter
def get_random_array():
    return np.random.randint(256, size = (1080,1920), dtype = np.uint8).astype(np.float64)

sample = get_random_array()
lazy_random_array = delayed(get_random_array)
lazy_arrays = [lazy_random_array() for i in range(2000)]
dask_arrays = [da.from_delayed(lazy_array, shape = sample.shape, dtype = sample.dtype) for lazy_array in lazy_arrays]
dask_stack = da.stack(dask_arrays, axis = 0)

viewer = napari.Viewer()
layer = viewer.add_image(data = dask_stack)

napari.run()

The above code results in many hundreds of calls to get_random_array() and overwhelms system memory/CPU. The likely culprit is the layer_utils calc_data_range() function, which nominally checks for array size before performing the min/max calculation. However, in the case of script above the entire array seems to be evaluated when deriving contrast limits.

Expected behavior

Correctly prevent evaluation of excessively large arrays for contrast limits check.

Environment

napari: 0.4.9 Platform: Windows-10-10.0.19041-SP0 Python: 3.8.10 | packaged by conda-forge | (default, May 11 2021, 06:25:23) [MSC v.1916 64 bit (AMD64)] Qt: 5.12.9 PyQt5: 5.12.3 NumPy: 1.20.2 SciPy: 1.6.3 Dask: 2021.05.0 VisPy: 0.6.6

OpenGL:

• GL version: 4.6.0 NVIDIA 462.31
• MAX_TEXTURE_SIZE: 32768

Screens:

• screen 1: resolution 2560x1440, scale 1.0
• screen 2: resolution 1440x2560, scale 1.0

Plugins:

• console: 0.0.3
• scikit-image: 0.4.9.dev8+g43a604be
• svg: 0.1.5

[jni]

Maybe setting the default contrast limit to the maximum values allowed by the datatype for large images/dask arrays would be a possible solution?

Thanks for the suggestion @jo-mueller! We already do this for uint8 data. For float data it is clearly problematic. For uint16, it’s also frequently a problem because 12-bit data in [0, 4096) is often packed into a uint16, resulting in vastly reduced contrast. But yes, we discussed this at the dev meeting and we are going to turn off auto-computation of contrast limits for anything that’s not a NumPy array. Instead we will make it easier to reset the contrast limits based on in-memory data by adding a “Reset” button to the contrast limits slider.

[sofroniewn]

@jwindhager agreed - and just in case anyone coming here is wondering what the behaviour is, it is performed when each layer is added separately, and so if you make three calls to

viewer.add_image(data_c1)
viewer.add_image(data_c2)
viewer.add_image(data_c3)

you’ll get 3 layers and it will happen 3 times. Or

viewer.add_image(data_all_c, channel_axis=0)

you’ll get 3 layers and it will happen 3 times. But

viewer.add_image(data_all_c)

you’ll only get one layer and it will just happen once, but now all three channels will use same colormap and so be forced to have some contrast_limits.