Can't evaluate model retinanet_R_50_FPN_3x on COCO

1. what changes you made (git diff) or what code you wrote No changes like the tutorials

2. what exact command you run: register_coco_instances(“my_test”, {}, “dataset/test.json”, “dataset/test/images”) evaluator = COCOEvaluator(“my_test”, cfg, False, output_dir=“./output/”) val_loader = build_detection_test_loader(cfg, “my_test”) inference_on_dataset(trainer.model, val_loader, evaluator)

3. what you observed (including the full logs): I trained the model on my own dataset. With Faster R-CNN models the evaluation feature works with NO problems and delivers metrics

If I train with retina config and retina weights cfg.MODEL.WEIGHTS = “detectron2://COCO-Detection/retinanet_R_50_FPN_3x/137849486/model_final_4cafe0.pkl”

The same code that works for Faster R-CNN: delivers an error in RetinaNet: [12/04 18:05:36 d2.evaluation.coco_evaluation]: Preparing results for COCO format …

~/detectron2/detectron2/evaluation/coco_evaluation.py in _eval_predictions(self, tasks) 147 } 148 for result in self._coco_results: –> 149 result[“category_id”] = reverse_id_mapping[result[“category_id”]] 150 151 if self._output_dir:

KeyError: 39

I only have one class. But there is no problem with the category_id (i’m starting on 1) as it works with Faster R-CNN with the same json files. As a comment, Retina has no ROI_heads so we have to use different threshold parameter cfg.MODEL.RETINANET.SCORE_THRESH_TEST = 0.4

This is the same error documented here

Environment

detectron2 has been successfully installed.

[ebrahim31]

@shamim-io Maybe try cfg.MODEL.RETINANET.NUM_CLASSES = 1?

[ppwwyyxx]

Your model predicts classes 39 that’s not in your dataset. You may have missed https://detectron2.readthedocs.io/tutorials/datasets.html#update-the-config-for-new-datasets