Why I got the map value -1

❓ Questions and Help

After training on coco2017 dataset, I got the following output:

2018-10-26 07:57:49,972 maskrcnn_benchmark.inference INFO: Total inference time: 0:30:09.844855 (0.08900146815007108 s / img per device, on 2 devices)
2018-10-26 07:57:57,928 maskrcnn_benchmark.inference INFO: Preparing results for COCO format
2018-10-26 07:57:57,928 maskrcnn_benchmark.inference INFO: Preparing bbox results
2018-10-26 07:58:06,302 maskrcnn_benchmark.inference INFO: Evaluating predictions
Loading and preparing results...
DONE (t=7.88s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=84.73s).
Accumulating evaluation results...
DONE (t=22.14s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
2018-10-26 08:00:20,193 maskrcnn_benchmark.inference INFO: OrderedDict([('bbox', OrderedDict([('AP', -1.0), ('AP50', -1.0), ('AP75', -1.0), ('APs', -1.0), ('APm', -1.0), ('APl', -1.0)]))])

I used the file e2e_faster_rcnn_R_50_FPN_1x.yaml, and modified the lr to 0.01.

[fmassa]

Hi,

In order to reproduce the results on fewer GPUs than 8, you’ll need indeed to change the learning rate (which is good in your case), but also the number of iterations should be increased from the default by a factor of 4x, as well as the learning rate schedules. So you should have 90000 * 4= 360000 iterations, and you need to change the lr schedules to be [240000, 320000].

Check the README on the single GPU training sections for more informations.

I’m closing the issue ad it doesn’t seem to be a bug, but please let me know if you have other questions

[auroua]

I followed you advice and got the following results on coco2017 val dataset:

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.371
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.587
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.405
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.216
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.401
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.485
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.309
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.483
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.508
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.318
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.542
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.643

Thanks!