Incorrect number of masks returned after augmentation

🐛 Bug

Describe the bug Incorrect number of masks returned after augmentation. It happend when some boxes goes outside of image. Number of boxes changed but masks stays the same.

To Reproduce

from albumentations import *
from albumentations import __version__ as ver


transform_generator = Compose([
        ShiftScaleRotate(p=1.0, shift_limit=0.1, scale_limit=0.2, rotate_limit=45, border_mode=cv2.BORDER_REFLECT),
    ], bbox_params={'format': 'pascal_voc',
                    'min_area': 0,
                    'min_visibility': 0.0,
                    'label_fields': ['labels']}, p=1.0)


def albu_bug():
    print('Albu ver: {}'.format(ver))
    image = np.zeros((500, 500, 3), dtype=np.uint8)
    mask1 = np.zeros((500, 500, 3), dtype=np.uint8)
    mask2 = np.zeros((500, 500, 3), dtype=np.uint8)
    mask3 = np.zeros((500, 500, 3), dtype=np.uint8)
    mask1[0:10, 0:10] = 255
    mask2[-10:, -10:] = 255
    mask3[250:260, 250:260] = 255

    ann = dict()
    ann['image'] = image.copy()
    ann['masks'] = [mask1, mask2, mask3]
    ann['labels'] = np.array([0, 0, 0])
    ann['bboxes'] = np.array([[0, 0, 10, 10], [490, 490, 500, 500], [250, 250, 260, 260]])

    for i in range(10):
        augm = transform_generator(**ann)
        print(len(ann['masks']), len(ann['labels']), len(ann['bboxes']))
        print(len(augm['masks']), len(augm['labels']), len(augm['bboxes']))


if __name__ == '__main__':
    albu_bug()

Expected behavior Masks corrsponding to removed boxes must be also removed

Environment

• Albumentations version (e.g., 0.1.8): 0.5.2
• Python version (e.g., 3.7): 3.7
• OS (e.g., Linux): Win10
• How you installed albumentations (conda, pip, source): pip

[ChrisDAT20]

I face the same issue as @ZFTurbo: I have situations where the masks do not match the bboxes. The image has 25 annotations, bboxes and segmentations which are converted to masks using the COCO api (coco.annToMask(). Then the image and targets are passed through a few transforms, CenterCrop, Resize, Affine… The transformed dict only contains 9 boxes, but it still containes 25 masks. When I filter out masks which only contain zeros masks = [i for i in masks if i.any()], most of the time this works. But sometimes a bbox is still there where its mask is gone after I filtered “empty” masks.

…than create some strange code to augment both bboxes and mask and check them after each transformation.

I thought thats what DualTransforms for, applying the transforms to both image and target in the same way. But it seems that bbox and mask transforms might get inconsistent.

[Dipet]

So, if you have bbox like on example image it is will be incorrect after crop. For this reason I think better augment only masks, because they are much more correct. So if you will augment only mask and then create bboxes it is better than create some strange code to augment both bboxes and mask and check them after each transformation.