Getting error while feeding custom dataset

I want to train the reidentification model on VeRi vehicle dataset. I followed the instructions given here on how to use custom dataset: https://kaiyangzhou.github.io/deep-person-reid/user_guide.html#use-your-own-dataset

When I try to train “hacnn” model on this custom (VeRi) dataset, I am not able to train the model. Please find below the dataset statistics and the error that I am getting:

=> Loading train (source) dataset
=> Loaded VeRiDataset
  ----------------------------------------
  subset   | # ids | # images | # cameras
  ----------------------------------------
  train    |   575 |    37746 |        20
  query    |   200 |     1678 |        19
  gallery  |   200 |    11579 |        19
  ----------------------------------------
=> Loading test (target) dataset
=> Loaded VeRiDataset
  ----------------------------------------
  subset   | # ids | # images | # cameras
  ----------------------------------------
  train    |   575 |    37746 |        20
  query    |   200 |     1678 |        19
  gallery  |   200 |    11579 |        19
  ----------------------------------------


  **************** Summary ****************
  train            : ['veri_dataset']
  # train datasets : 1
  # train ids      : 575
  # train images   : 37746
  # train cameras  : 20
  test             : ['veri_dataset']
  *****************************************


=> Start training
/home/rajat/MyPC/DFKI/MasterThesis/Vehicle_Reidentification/veri/lib/python3.6/site-packages/torch/nn/functional.py:2457: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
  warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
Traceback (most recent call last):
  File "train_torchreid.py", line 52, in <module>
    print_freq=10
  File "/home/rajat/MyPC/DFKI/MasterThesis/Vehicle_Reidentification/veri/lib/python3.6/site-packages/torchreid-0.7.8-py3.6-linux-x86_64.egg/torchreid/engine/engine.py", line 100, in run
    self.train(epoch, max_epoch, trainloader, fixbase_epoch, open_layers, print_freq)
  File "/home/rajat/MyPC/DFKI/MasterThesis/Vehicle_Reidentification/veri/lib/python3.6/site-packages/torchreid-0.7.8-py3.6-linux-x86_64.egg/torchreid/engine/image/softmax.py", line 99, in train
    loss = self._compute_loss(self.criterion, outputs, pids)
  File "/home/rajat/MyPC/DFKI/MasterThesis/Vehicle_Reidentification/veri/lib/python3.6/site-packages/torchreid-0.7.8-py3.6-linux-x86_64.egg/torchreid/engine/engine.py", line 302, in _compute_loss
    loss = DeepSupervision(criterion, outputs, targets)
  File "/home/rajat/MyPC/DFKI/MasterThesis/Vehicle_Reidentification/veri/lib/python3.6/site-packages/torchreid-0.7.8-py3.6-linux-x86_64.egg/torchreid/losses/__init__.py", line 21, in DeepSupervision
    loss += criterion(x, y)
  File "/home/rajat/MyPC/DFKI/MasterThesis/Vehicle_Reidentification/veri/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/rajat/MyPC/DFKI/MasterThesis/Vehicle_Reidentification/veri/lib/python3.6/site-packages/torchreid-0.7.8-py3.6-linux-x86_64.egg/torchreid/losses/cross_entropy_loss.py", line 47, in forward
    targets = torch.zeros(log_probs.size()).scatter_(1, targets.unsqueeze(1).data.cpu(), 1)
RuntimeError: Invalid index in scatter at /pytorch/aten/src/TH/generic/THTensorEvenMoreMath.cpp:551

Note: The same code is working fine if I use “market1501” dataset instead of my custom one.

Please help me in solving this issue.

Thanks

[KaiyangZhou]

you need to relabel the training images so that the label starts from 0 and increments by 1 (think about how the classification layer is constructed and how a one-hot vector is generated given a label) e.g. raw_label=>new_label 1=>0 3=>1 4=>2 7=>3 8=>4

there is one simple way to convert the label

label_dict = {label: index for index, label in enumerate(labelset)}
new_label = label_dict[raw_label]

[youngjae-avikus]

@Rajat-Mehta

Could you tell me your custom_datset code?