Question about Classes Index

Thank you for sharing this great work! I have a question about your classes index. YoutubeVIS has 40 classes and a empty class is involved in VisTR, thus 41 classes in total, right? Then I noticed that in your implementation, the loss_labels() is computed by

target_classes = torch.full(src_logits.shape[:2], self.num_classes, dtype=torch.int64, device=src_logits.device)
target_classes[idx] = target_classes_o
loss_ce = F.cross_entropy(src_logits.transpose(1, 2), target_classes, self.empty_weight)

Since ytvis’s class begins from 1 to 40 and I didn’t find any re-index operation in your code, I am confused by the empty class here. Do you use 40 represent empty class (you set self.num_classes=40)? Thank you

[Epiphqny]

Hi @lxa9867, thanks for your question, I fixed the bug and changed the class number to 41, the result improves from 0.38 to 0.40, I will update the code and result later.

[Epiphqny]

hi @lxa9867, I have fixed the bug and updated the code and result.