question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

ndfilters.convolve wrap mode not working when the array is chunked

See original GitHub issue

What happened: Used ndfilters.convolve using the wrap mode and the output had the wrong values on the edges What you expected to happen: The function to output the correct values on the edges. Minimal Complete Verifiable Example:

import numpy as np
import dask.array as da
from dask_image.ndfilters import convolve


def test_correct(F,axis = 0):
    if axis !=0:
        F = da.swapaxes(F, 0, axis)
        result = test_correct(F,axis = 0)
        return result
    
    kernel = np.array([1,-1,0,1,-1]).reshape((5,) + (1,) * (F.ndim - 1))
    # The output using ndfilters.convolve with the wrap mode
    O1 = convolve(F,kernel, mode = 'wrap')
    # Output created by forcing the wrapping using dask.array.overlap.overlap then calculating the convolution. 
    # taking care of the boundary
    F_1 = da.overlap.overlap(F, depth = {axis : 2}, boundary = 'periodic')
    # calculate the convolution
    O2_int = convolve(F_1,kernel, mode = 'constant')
    # removing the useless ghosts zones. 
    O2 = da.overlap.trim_overlap(O2_int, depth = {axis : 2})

    return O1.compute(), O2.compute()

test_1D_no_chunks = da.from_array([1,2,3,4,5,6,7,8,9],chunks = (-1,))
test_1D_chunked = da.from_array([1,2,3,4,5,6,7,8,9],chunks = (5,))

print('1D no chunks :',*test_correct(test_1D_no_chunks), sep = '\n', end = '\n\n')
print('1D chunked :', *test_correct(test_1D_chunked), sep = '\n', end = '\n\n')

test_2D_no_chunks = da.from_array(np.arange(25).reshape(5,5),chunks =(-1,-1))
test_2D_chunked = da.from_array(np.arange(25).reshape(5,5),chunks =(-1,2))

print('2D no chunks axis 0  :',*test_correct(test_2D_no_chunks,axis = 0), sep = '\n\n', end = '\n\n\n')
print('2D no chunks axis 1  :',*test_correct(test_2D_no_chunks,axis = 1), sep = '\n\n', end = '\n\n\n')

print('2D chunked axis 0  :',*test_correct(test_2D_chunked,axis = 0), sep = '\n\n', end = '\n\n\n')
print('2D chunked axis 1  :',*test_correct(test_2D_chunked,axis = 1), sep = '\n\n', end = '\n\n\n')
1D no chunks :
[ 2 -7  2  2  2  2  2 -7  2] # ndfilters.convolve
[ 2 -7  2  2  2  2  2 -7  2] # expected result

1D chunked :
[ 2 -5  2  2  2  2  2 -4  2] # ndfilters.convolve
[ 2 -7  2  2  2  2  2 -7  2] # expected result

2D no chunks axis 0  :

[[ 10  10  10  10  10]
 [-15 -15 -15 -15 -15]
 [ 10  10  10  10  10]
 [-15 -15 -15 -15 -15]
 [ 10  10  10  10  10]] # ndfilters.convolve

[[ 10  10  10  10  10] 
 [-15 -15 -15 -15 -15]
 [ 10  10  10  10  10]
 [-15 -15 -15 -15 -15]
 [ 10  10  10  10  10]] # expected result


2D no chunks axis 1  :

[[ 2  2  2  2  2]
 [-3 -3 -3 -3 -3]
 [ 2  2  2  2  2]
 [-3 -3 -3 -3 -3]
 [ 2  2  2  2  2]] # ndfilters.convolve

[[ 2  2  2  2  2]
 [-3 -3 -3 -3 -3]
 [ 2  2  2  2  2]
 [-3 -3 -3 -3 -3]
 [ 2  2  2  2  2]] # expected result


2D chunked axis 0  :

[[ 10  10  10  10  10]
 [-15 -15 -15 -15 -15]
 [ 10  10  10  10  10]
 [-15 -15 -15 -15 -15]
 [ 10  10  10  10  10]] # ndfilters.convolve

[[ 10  10  10  10  10]
 [-15 -15 -15 -15 -15]
 [ 10  10  10  10  10]
 [-15 -15 -15 -15 -15]
 [ 10  10  10  10  10]] # expected result


2D chunked axis 1  :

[[ 2  2  2  2  2]
 [-2 -2 -2 -2 -2]
 [ 2  2  2  2  2]
 [-3 -3 -3 -3 -3]
 [ 2  2  2  2  2]] # ndfilters.convolve

[[ 2  2  2  2  2]
 [-3 -3 -3 -3 -3]
 [ 2  2  2  2  2]
 [-3 -3 -3 -3 -3]
 [ 2  2  2  2  2]] # expected result

Anything else we need to know?:

The workaround is used there to get the correct value might be a good way to fix this bug.

Environment:

  • Dask version: dask 2021.4.1; dask-core 2021.4.1; dask-image 0.6.0
  • Python version: 3.9.5
  • Operating System: Ubuntu 20
  • Install method (conda, pip, source): dask and dask-core: pip; dask-image: conda-forge

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:9 (8 by maintainers)

github_iconTop GitHub Comments

2reactions
anlavandiercommented, Sep 29, 2021

The issue is now closed with the merging of #243

1reaction
GenevieveBuckleycommented, Sep 22, 2021

Thank you for the report!

Read more comments on GitHub >

github_iconTop Results From Across the Web

scipy.ndimage.convolve — SciPy v1.9.3 Manual
Multidimensional convolution. The array is convolved with the given kernel. ... The mode parameter determines how the input array is extended beyond its ......
Read more >
python - scipy.ndimage.filters.convolve and multiplying Fourier ...
This turned out to be a fascinating question. It seems that convolution using the Discrete Fourier Transform (as implemented by ...
Read more >
Implement non-axis windows and chunks iterators #276 - GitHub
Today I looked into the ndarray code to wrap my head around the iterator and especially the BaseIterator and ArrayBase code. It is...
Read more >
dask-image Documentation
Wrapped copy of “scipy.ndimage.filters.convolve”. Excludes the output parameter as it would not work with Dask arrays. Original docstring:.
Read more >
nd Documentation
Most of the algorithms work on both Dataset and DataArray objects. ... In the first instance, the change detection problem then becomes a...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found