Cannot eval on COCO test-dev on Google Colab

I am trying to use YOLACT++ from Google Colab. Inference on images/videos works fine, but I can’t evaluate on COCO test-dev. I run this command: python eval.py --trained_model=weights/yolact_plus_resnet50_54_800000.pth --dataset=coco2017_testdev_dataset And I get:

Traceback (most recent call last):
 File "eval.py", line 1105, in <module>
  evaluate(net, dataset)
 File "eval.py", line 936, in evaluate
  img, gt, gt_masks, h, w, num_crowd = dataset.pull_item(image_idx)
 File "/content/yolact/data/coco.py", line 172, in pull_item
  if target.shape[0] == 0:
AttributeError: 'NoneType' object has no attribute 'shape'

If I change line 172 in coco.py to check if target is None I get another error:

Traceback (most recent call last):
  File "eval.py", line 1105, in <module>
    evaluate(net, dataset)
  File "eval.py", line 956, in evaluate
    prep_metrics(ap_data, preds, img, gt, gt_masks, h, w, num_crowd, dataset.ids[image_idx], detections)
  File "eval.py", line 390, in prep_metrics
    gt_boxes = torch.Tensor(gt[:, :4])
TypeError: 'NoneType' object is not subscriptable

I can get past this error by adding the argument --output_coco_json, but I don’t think it’s the right way to use the script. I think it’s a bug but could also be a Colab related issue (or some error of mine).

[dbolya]

You can use --score_threshold=0.05 to reduce the file size. As for the first bug, that’s an oversight on my part. For now you can comment it out. I’ll try to get a fix out today.

[ShihuaHuang95]

i got the same error, how to solves?

Comment these lines: https://github.com/dbolya/yolact/blob/092554ad707c2749631dfe545c8a953b2b3f4a68/data/coco.py#L172-L174

and use --output_coco_json flag.

For me, I add another condition based on the previous one as below: “if target.shape[0] == 0 and self.has_gt:”