Log evaluation loss while testing model

I would love to see the loss for the test set while evaluating the mode during training. I have modified code to return loss when testing,

class SSDDetector(nn.Module):
    def __init__(self, cfg):
        super().__init__()
        self.cfg = cfg
        self.backbone = build_backbone(cfg)
        self.box_head = build_box_head(cfg)

    def forward(self, images, targets=None):
        features = self.backbone(images)
        detections, detector_losses = self.box_head(features, targets)
        if self.training:
            return detector_losses
        return detections, detector_losses

However, detector_losses are empty during evaluation or validation. Your suggestions will be most helpful.

[priteshgohil]

Finally, I can get validation loss after the following edit. The reason was that, network outputs normalized bounding box and dataloader outputs (target label) was not normalized (value of target cords was too high). Therefore error value is quite large for regression loss.

Following are the additional changes in https://github.com/lufficc/SSD/blob/50373c79b861d5d239be4206fafc6661cea040b4/ssd/data/transforms/__init__.py#L20

    else:
        transform = [
            ConvertFromInts(), # Convert img to float32
            ToPercentCoords(), # Normalize BBox Cords
            Resize(cfg.INPUT.IMAGE_SIZE),
            SubtractMeans(cfg.INPUT.PIXEL_MEAN),
            ToTensor()
        ]

And validation loss looks good,

[lufficc]

SSD won’t compute the losses when testing since it’s unnecessary and time consuming. But if you just want to see, here are some suggestions:

1. compute targets when testing by pass target_transform: https://github.com/lufficc/SSD/blob/master/ssd/data/build.py#L32

2. remember pass to model: https://github.com/lufficc/SSD/blob/master/ssd/engine/inference.py#L43

3. compute loss when testing here https://github.com/lufficc/SSD/blob/master/ssd/modeling/box_head/box_head.py#L39 follows https://github.com/lufficc/SSD/blob/master/ssd/modeling/box_head/box_head.py#L31-L34