Lower accuracy than expected on VOC 2007 model & data

Thank you for providing your code. I have installed and run the test provided but unfortunately I am seeing lower accuracy on the VOC 2007 benchmark than I expected.

On the Readme I see that the model achieves 71.2 but when I run ./experiments/scripts/test_vgg16.sh 0 pascal_voc with VOC 2007 data and your model I see a result of Mean AP = 0.4955. If I am right this is meant to be interpreted as an mAP of 49.55. Should I be using a different testing script or different model than the one downloaded by ./data/scripts/fetch_faster_rcnn_models.sh ? Here are the full results:

AP for aeroplane = 0.5898 AP for bicycle = 0.5308 AP for bird = 0.4317 AP for boat = 0.3876 AP for bottle = 0.2347 AP for bus = 0.6052 AP for car = 0.5414 AP for cat = 0.6908 AP for chair = 0.2789 AP for cow = 0.5222 AP for diningtable = 0.5555 AP for dog = 0.6149 AP for horse = 0.7065 AP for motorbike = 0.5160 AP for person = 0.4421 AP for pottedplant = 0.2304 AP for sheep = 0.4441 AP for sofa = 0.5538 AP for train = 0.6770 AP for tvmonitor = 0.3559 Mean AP = 0.4955

[mbuckler]

Hey! Thank you very much for replying. As it turns out the reason for the incorrect accuracy was due to the wrong GPU architecture being specified in the extra_compile_args in the setup.py script. I needed -arch=sm+52 because I am using a Titan X. Interestingly the segfault still appears at the end of testing, but since the testing is functional I have all that I need to continue.

I’ll close this issue for sure since everything is working, but I should also mention that a few things might want to be changed in the code to help Linux users like myself. I’ve made a pull request that you might be interested in. Thanks again!

[endernewton]

Not sure what happened in your case. Can you provide your hardware and software setup? I don’t have segfault here.