GDumb and data augmentation

Hi! I’m testing some strategies to reach the accuracy values stated in the original papers. In particular, Gdumb with mem_size=500 reaches 90% of accuracy in the split-mnist benchmark. For the parameters I use the original implementation of GDumb: https://github.com/drimpossible/GDumb

🐛 Describe the bug There is a drop (~30%) in accuracy using data augmentation. This problem doesn’t appear using other strategies (with the same settings: neural network, parameters, regularization etc.) .

🐜 To Reproduce

train_transform = Compose([
        #RandomCrop(28, padding=4),
        ToTensor(),
        Normalize((0.1307,), (0.3081,))
    ])
eval_transform = Compose([
        ToTensor(),
        Normalize((0.1307,), (0.3081,))
    ])

scenario = SplitMNIST(
        n_experiences=5, fixed_class_order=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
        seed=1234, train_transform=train_transform, eval_transform=eval_transform)

🐝 Expected behavior Data augmentation should help to achieve better results despite the strategy.

[AntonioCarta]

@gab709 I did some experiments. I don’t think there is any bug here. The problem is that you are using an MLP. If you use a CNN everything works as expected.

Here are the results using a LeNet-like CNN:

┏━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Exp/Task ┃ Top1_Acc_Exp ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ E0T0     │ 0.8178       │
│ E1T0     │ 0.9759       │
│ E2T0     │ 0.7593       │
│ E3T0     │ 0.9310       │
│ E4T0     │ 0.6485       │
└──────────┴──────────────┘

[AntonioCarta]

@gab709 if you want to look into this, I can support you. Just ping me on slack if you need help.

Otherwise, I will need the complete script to reproduce the error.