Documentation for ConfusionMatrix is thin on details, and error messages not informative

🐛 Bug description

Confusion matrix requires that y_pred and y are to have different dimensions (batch,num_classes) and (batch,), respectively. Getting this wrong results in the following error message, which makes no sense (batch size is 5 here, and both y_pred and y have shape (5,1):

y_pred must have shape (batch_size, num_categories, ...) and y must have shape of (batch_size, ...), but given torch.Size([5, 1]) vs torch.Size([5, 1])

I believe that the torch.Size([5, 1]) vs torch.Size([5, 1]) message should probably read torch.Size([5, 1]) vs torch.Size([5,]) to illustrate the problem.

Separately, it would be good to mention in the ConfusionMatrix docs that one-hot encoded vectors are required, and that this can be implemented using from ignite.utils.to_onehot (I stumbled upon it after realizing I had to one-hot encode, since most people use BCEWithLogitsLoss() anyway, and hence their model predictions are not one-hot encoded).

Environment

• PyTorch Version (e.g., 1.4): 1.4.0
• Ignite Version (e.g., 0.3.0): 0.4.2
• OS (e.g., Linux): MacOS
• How you installed Ignite (conda, pip, source): pip
• Python version: 3.8
• Any other relevant information:

[vfdev-5]

@nsinenian thanks for the details. Yes, that’s true that you have binary classification use-case and yes, it definitely makes sense to mention that how to deal with that in the docs !

[nsinenian]

@vfdev-5 The comment was with respect to predictions, not targets. For the predictions, I have (relevant excerpt):

__NUM_CLASSES__ = 2

def binary_one_hot_output_transform(output):
    """ Ouptuts a one-hot encoded vector with 0 or 1 values """
    [y_pred,y] = output
    [y_pred,y] = [torch.sigmoid(y_pred).round().long(),y]    
    return (to_onehot(y_pred,__NUM_CLASSES__), y.long())

valuation_metrics = {
        'confusion_matrix' : ConfusionMatrix(num_classes=__NUM_CLASSES__,output_transform=binary_one_hot_output_transform),
}

evaluator = create_supervised_evaluator(model,metrics=valuation_metrics,output_transform=lambda x,y,y_pred: (y_pred,y))
trainer = create_supervised_trainer(model,optimizer,criterion)

My model output (forward function) is:

    def forward(self,features):
      
        y = self.fc1(features)
        y = self.fcd(y)
        y = self.fc2(F.relu(y))

        return y

For binary classification, I use the BCEWithLogitsLoss() loss function which does not use one-hot encoded vectors (and hence the model output being what it is above). Perhaps my use case is atypical.