Different performance migrating from torchvision to Albumentations for this transformation

🐛 Bug

Hi, I have been trying to reproduce the augmentations from SimCLR/MocoV2 from torchvision in Albumentations (see https://github.com/facebookresearch/moco/blob/master/main_moco.py#L225 reproduced below). Unfortunately, I have not been able to get nearly the same performance after replacing the torchvision pipeline with the below Albumentations version. Could someone point me to how these are different? I suspect it’s related to ColorJitter - it would be very helpful for it to be clear what the drop-in Albumentations replacement should be, I followed https://github.com/albumentations-team/albumentations/issues/672 but it is perhaps related to #690 . (It may also be GaussianBlur related, but I don’t think so.) I’m not sure how to verify identical transformations across both given the stochasticity and how some aren’t visually apparent, aside from performance on the task.

To Reproduce

Steps to reproduce the behavior: torchvision ver:

augmentation = [
            transforms.RandomResizedCrop(224, scale=(0.2, 1.)),
            transforms.RandomApply([
                transforms.ColorJitter(0.4, 0.4, 0.4, 0.1) 
            ], p=0.8),
            transforms.RandomGrayscale(p=0.2),
            transforms.RandomApply([GaussianBlur([.1, 2.])], p=0.5),
            transforms.RandomHorizontalFlip(),
            transforms.ToTensor(),
            normalize
        ]

where GaussianBlur is defined as

from PIL import ImageFilter
import random

class GaussianBlur(object):
    """Gaussian blur augmentation in SimCLR https://arxiv.org/abs/2002.05709"""

    def __init__(self, sigma=[.1, 2.]):
        self.sigma = sigma

    def __call__(self, x):
        sigma = random.uniform(self.sigma[0], self.sigma[1])
        x = x.filter(ImageFilter.GaussianBlur(radius=sigma))
        return x

Albumentations ver:

normalization_mean = [0.485, 0.456, 0.406] #np.array([0.485, 0.456, 0.406])
normalization_std = [0.229, 0.224, 0.225] #np.array([0.229, 0.224, 0.225])
Anormalize = A.Normalize(mean=normalization_mean, std=normalization_std, max_pixel_value=1) 

 augmentation = [
                A.RandomResizedCrop(height, height, scale=(0.2, 1.)),
                A.Compose([
                    A.RandomBrightnessContrast(0.4, 0.4, p=1),
                    A.HueSaturationValue(hue_shift_limit=int(0.1*255), sat_shift_limit=int(0.4*255), val_shift_limit=0, p=1), # 
                ], p=0.8), # https://github.com/albumentations-team/albumentations/issues/672
                A.ToGray(p=0.2),
                A.GaussianBlur(blur_limit=(23, 23), sigma_limit=(0.1,2.), p=0.5), 
                A.HorizontalFlip(),
                A.ToFloat(255),
                Anormalize,
                ToTensorV2(),
            ]

Expected behavior

Equivalent transformations.

Environment

• Albumentations version (e.g., 0.1.8): 0.4.6
• Python version (e.g., 3.7): 3.8.3
• OS (e.g., Linux): Ubuntu
• How you installed albumentations (conda, pip, source): pip
• Any other relevant information:

Additional context

Thank you!

[sachit-menon]

I tried changing hue_shift_limit to int(0.8*180) instead and unfortunately it did not really improve. Inspired by this though, I removed color jittering from both the torchvision and albumentations versions and got similar performance, which suggests the difference probably is in there somehow. Can you think of anything else that could be different about the way I’ve tried to do color jittering with Albumentations?

If I understand about ToFloat, it’s not needed if you set max_pixel_value of A.Normalize to 255 instead of 1 like I did - I just found this more readable since the torchvision Normalize assumes max of 1, since ToTensor divides.

In case it’s relevant: instead of loading with opencv, I’m using the default torchvision loading with Pillow and am applying the Albumentations transformations as a drop-in replacement (after casting to np.array), could that cause discrepancies? I’m also seeing a slowdown of about 2x when using Albumentations instead of torchvision that I wonder could be related to data loading.

[Dipet]

You also can try to change HueSaturationValue by set flag brightness_by_max=False. Converting image from Pillow to albumentations reduce the speed, but I do not know how much.

Try to remove GaussianBlur and ColorJitter, if results will be close, try after that to add CollorJitter. If problem not in ColorJitter, I think problem may be only in GaussianBlur