Patch extraction bug, or unclear functionality?

branch: unit_tests commit: 9b8fd6d

I encountered something that looks to me like a bug, but it’s possible that it’s not clear to me the exact intended functionality of extract patch.

I found this while writing tests for extract_patch. If you pull the branch, you can encounter the suspected issue by calling py.test --cov . -s -m current (will run that single test here that I decorated with @pytest.mark.current.

What I’m doing is setting up a fake 16x16 image,

[[ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15]

 [ 1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16]

 [ 2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17]

 [ 3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18]

 [ 4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19]

 [ 5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20]

 [ 6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21]

 [ 7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22]

 [ 8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23]

 [ 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24]

 [10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25]

 [11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26]

 [12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27]

 [13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28]

 [14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29]

 [15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30]]

and want to extract patches of size 4 (4x4).

In the tests, I expected that the first patch was the first 4x4 square (top left), which was correct. Same for the top right patch, which is the 4th array of the output of extract_patch.

I encountered the error testing the 5th array of the output of extract_patch, which I expected to be the left-most 4x4 of “second row” of 4x4 blocks (e.g. self.testImage[4:8,0:4]), that is:

[[ 4  5  6  7]
[ 5  6  7  8]
[ 6  7  8  9]
[ 7  8  9 10]]

However, the extract_patch function is actually creating an overlap of the top row:

[[ 3  4  5  6]
[ 4  5  6  7]
[ 5  6  7  8]
[ 6  7  8  9]]

and throwing an error. The same is true for the top-most 4x4 square of the second column (not included in the test), meaning the 2nd array of the output of extract_patch.

Is there a reason to overlap rows/columns for a patch even the patch is an integer factor of the square image size? I understand the need for overlap for other cases (e.g. non-square images, or non-integer fraction of image size / patch size), but what I expected here was 16 4x4 patches since the image is 16x16, however because of the overlap it resulted in 25 patches.

edit: Fixed the patch output example.

[perone]

We should have overlap during training, a configurable overlap would be even better because this is an hyper-param of the training procedure.

[alzaia]

@mathieuboudreau Ok so I think it is better to change the implementation IMO. This 1 pixel overlap in not intuitive and could cause issues in the future. I will look into this. Thank you for pointing it out.