Standard deviation for transforms.Normalize

Hey,

How did you calculate the standard deviation values for transforms.Normalize? I am getting the same means, but different standard deviations:

import numpy as np

from torchvision import datasets
from torchvision import transforms

transform_train = transforms.Compose([
#     transforms.RandomCrop(32, padding=4),
#     transforms.RandomHorizontalFlip(),
    transforms.ToTensor()
])

trainset = datasets.CIFAR10(root='data', train=True, download=True, transform=transform_train)
train_loader = torch.utils.data.DataLoader(trainset, batch_size=50_000, shuffle=True)

train = train_loader.__iter__().next()[0]

print('Mean: {}'.format(np.mean(train.numpy(), axis=(0, 2, 3))))
# Mean: [ 0.49139765  0.48215759  0.44653141]
print('STD: {}'.format(np.std(train.numpy(), axis=(0, 2, 3))))
# STD: [ 0.24703199  0.24348481  0.26158789]

[kuangliu]

See get_mean_and_std(dataset) in utils.py. It’s OK to use a different std for training.

[renyiryry]

Should be [0.2470, 0.2435, 0.2616]. See here.

Not really. The way you computed gives the std of the tensor of size [50000, 32, 32]. However, the desired quantity is the “averaged” std across all the images. Also, when computing the std for a single image, degree of freedom should be 1. (See https://github.com/kuangliu/pytorch-cifar/blob/master/utils.py)