CIFAR10 and SimpleCNN training: IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

🐛 Describe the bug Trying to modify notebook 4_training to fit SplitCIFAR10 with EWC in tutorials results in the following error

🐜 To Reproduce Steps / minimal snipped of code to reproduce the issue.

from avalanche.benchmarks.classic import SplitCIFAR10
from torch.nn import CrossEntropyLoss
from avalanche.models import SimpleCNN
from torch.optim import SGD

scenario = SplitCIFAR10(n_experiences=2, return_task_id=True)

criterion = CrossEntropyLoss()
model = SimpleCNN(num_classes=5)
optimizer = SGD(model.parameters(), lr=0.001, momentum=0.9)

strategy = EWC(model, optimizer = optimizer, criterion = criterion, ewc_lambda = 0.1)


# TRAINING LOOP
print('Starting experiment...')
results_ewc = []
for experience in scenario.train_stream:
    print(type(experience.dataset))
    print("Start of experience: ", experience.current_experience)
    
    print("shape dataset ", len(experience.dataset))
    
    print("Current Classes: ", experience.classes_in_this_experience)

    strategy.train(experience)
    print('Training completed')

    print('Computing accuracy on the whole test set')
    results_ewc.append(strategy.eval(scenario.test_stream))

🐝 Expected behavior A clear and concise description of what you expected to happen.

Training proceeds through the experiences

🐞 What happens

Starting experiment...
<class 'avalanche.benchmarks.utils.avalanche_dataset.AvalancheSubset'>
Start of experience:  0
shape dataset  25000
Current Classes:  [0, 1, 2, 3, 4]
-- >> Start of training phase << --
-- Starting training on experience 0 (Task 0) from train stream --
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-60-4c8d2621329b> in <module>
     12     print("Current Classes: ", experience.classes_in_this_experience)
     13 
---> 14     strategy.train(experience)
     15     print('Training completed')
     16 

~/miniconda3/envs/torch/lib/python3.8/site-packages/avalanche/training/strategies/base_strategy.py in train(self, experiences, eval_streams, **kwargs)
    247         self.before_training(**kwargs)
    248         for exp in experiences:
--> 249             self.train_exp(exp, eval_streams, **kwargs)
    250         self.after_training(**kwargs)
    251 

~/miniconda3/envs/torch/lib/python3.8/site-packages/avalanche/training/strategies/base_strategy.py in train_exp(self, experience, eval_streams, **kwargs)
    277         for self.epoch in range(self.train_epochs):
    278             self.before_training_epoch(**kwargs)
--> 279             self.training_epoch(**kwargs)
    280             self._periodic_eval(eval_streams, do_final=False)
    281             self.after_training_epoch(**kwargs)

~/miniconda3/envs/torch/lib/python3.8/site-packages/avalanche/training/strategies/base_strategy.py in training_epoch(self, **kwargs)
    431 
    432                 # Loss & Backward
--> 433                 self.loss += self.criterion(self.logits, self.mb_y)
    434 
    435             self.before_backward(**kwargs)

~/miniconda3/envs/torch/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    887             result = self._slow_forward(*input, **kwargs)
    888         else:
--> 889             result = self.forward(*input, **kwargs)
    890         for hook in itertools.chain(
    891                 _global_forward_hooks.values(),

~/miniconda3/envs/torch/lib/python3.8/site-packages/torch/nn/modules/loss.py in forward(self, input, target)
   1045     def forward(self, input: Tensor, target: Tensor) -> Tensor:
   1046         assert self.weight is None or isinstance(self.weight, Tensor)
-> 1047         return F.cross_entropy(input, target, weight=self.weight,
   1048                                ignore_index=self.ignore_index, reduction=self.reduction)
   1049 

~/miniconda3/envs/torch/lib/python3.8/site-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
   2691     if size_average is not None or reduce is not None:
   2692         reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 2693     return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
   2694 
   2695 

~/miniconda3/envs/torch/lib/python3.8/site-packages/torch/nn/functional.py in log_softmax(input, dim, _stacklevel, dtype)
   1670         dim = _get_softmax_dim("log_softmax", input.dim(), _stacklevel)
   1671     if dtype is None:
-> 1672         ret = input.log_softmax(dim)
   1673     else:
   1674         ret = input.log_softmax(dim, dtype=dtype)

IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

[vlomonaco]

Thanks @RgMoller for reporting the issue, I’ll look into it ASAP.

[AndreaCossu]

Mmmm… maybe it is because now the training progresses and reaches the second experience where you have labels in [5-9]. You should set the num_classes in SimpleCNN to 10 if you are training on CIFAR-10.

Let me know if this was the problem, otherwise I can look into this more carefully.