Passing cupy functions into cupyx.scipy.ndimage.filters.generic_filter causes a TypeError

It looks like you can’t pass cupy functions (like cupy.mean) into cupyx.scipy.ndimage.filters.generic_filter(). This is at odds with what I’d expected, based on how it works with numpy.

I have minimal working example here and it’s obvious I need to replace cupy.mean, I’m just not sure what I should replace it with. Does anyone have suggestions?

• Conditions (you can just paste the output of python -c 'import cupy; cupy.show_config()')

CuPy Version          : 8.0.0rc1
CUDA Root             : C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2
CUDA Build Version    : 10020
CUDA Driver Version   : 10020
CUDA Runtime Version  : 10020
cuBLAS Version        : 10202
cuFFT Version         : 10102
cuRAND Version        : 10102
cuSOLVER Version      : (10, 3, 0)
cuSPARSE Version      : 10301
NVRTC Version         : (10, 2)
Thrust Version        : 100907
CUB Build Version     : 100800
cuDNN Build Version   : None
cuDNN Version         : None
NCCL Build Version    : None
NCCL Runtime Version  : None
cuTENSOR Version      : None

• Code to reproduce

import numpy as np
import scipy.ndimage.filters

# this works
numpy_array = np.random.random((20, 20))
numpy_result = scipy.ndimage.filters.generic_filter(numpy_array, np.mean, 3, mode="reflect", cval=0)

import cupy
import cupyx.scipy.ndimage.filters

# this doesn't work, because cupy.mean is neither 
cupy_array = cupy.random.random((20, 20))
cupy_result = cupyx.scipy.ndimage.filters.generic_filter(cupy_array, cupy.mean, 3, mode="reflect", cval=0)

• Error messages, stack traces, or logs

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-8789e403f984> in <module>
      3 
      4 cupy_array = cupy.random.random((20, 20))
----> 5 cupy_result = cupyx.scipy.ndimage.filters.generic_filter(cupy_array, cupy.mean, 3, mode="reflect", cval=0)

c:\users\genevieve\anaconda3\envs\cupy-dask-image\lib\site-packages\cupyx\scipy\ndimage\filters.py in generic_filter(input, function, size, footprint, output, mode, cval, origin)
   1075                                                      mode, origin, 'footprint')
   1076     in_dtype = input.dtype
-> 1077     sub = _filters_generic._get_sub_kernel(function)
   1078     if footprint.size == 0:
   1079         return cupy.zeros_like(input)

c:\users\genevieve\anaconda3\envs\cupy-dask-image\lib\site-packages\cupyx\scipy\ndimage\_filters_generic.py in _get_sub_kernel(f)
     26         raise TypeError('only ReductionKernel allowed (not ElementwiseKernel)')
     27     else:
---> 28         raise TypeError('bad function type')
     29 
     30 

TypeError: bad function type

[coderforlife]

We are planning to support these eventually through fusion kernels (automatically create CUDA kernels from Python code), however those are significantly more tricky to implement so are currently not supported.

[moataz-elashery]

The functions that are passed to generic_filter will be called inside the GPU, so they need to be actual CUDA kernels, there is no easy way that we can call python code or regular host code from the GPU. You will need to create your own custom RawKernel or ReductionKernel.

Inside a ReductionKernel you can’t call a regular cupy function as you are trying now, so you need to implement the mean yourself 😞.

This is a snippet that does what you want. Hope you find it useful.

import cupy
import cupyx.scipy.ndimage.filters

my_mean = cupy.ReductionKernel(
    'T x',  # input params
    'T y',  # output params
    'x',  # map
    'a + b',  # reduce
    'y = a / _in_ind.size()',  # This is an undocumented variable and a hack
    '0',  # identity value
    'mean'  # kernel name
)

cupy_array = cupy.random.random((20, 20))
cupy_result = cupyx.scipy.ndimage.filters.generic_filter(cupy_array, my_mean, 3, mode="reflect", cval=0)

Hello I need to use nanmean along with the cupyx scipy generic_filter But I don’t know how to make the custom function (how to count the nan present and ignore them )

I am making this filter to fill gaps in the images with the mean of the neighboring pixels ignoring the empty neighbors After this step, I will use boolean indexing to extract the new values of the previously empty cells leaving the original pixels as they were.

I’ll attach my numpy code for illustration

#input:
array= np.array([[[13, 21, 13,  np.nan],
                 [ 5, np.nan, 22, 14],
                 [15, 33,  np.nan,  np.nan],
                 [ 0,  0, np.nan,  np.nan]],
                [[13, 17, 13,  np.nan],
                 [ 5, np.nan, 33, 14],
                 [21, 33,  np.nan,  np.nan],
                 [ 0,  0, np.nan,  np.nan]],                
                [[13, 21, 13,  np.nan],
                 [ 5, np.nan, 70, 14],
                 [21,  np.nan,  np.nan,  np.nan],
                  [ 0,  0, np.nan,  np.nan]]]
                , dtype=np.float32) 

#footprint
mask = np.zeros((3,3,3))
mask[1,:]=1
mask[1,1, 1] = 0

result= ndimage.generic_filter(array, np.nanmean, footprint=mask, mode='constant', cval=np.nan)

Thanks for your time

@emcastillo @coderforlife