The nms seems not working?

Thanks for your nice code. In visualize.py I changed the code: retinanet = model.resnet50(num_classes=dataset_val.num_classes(),) retinanet.load_state_dict(torch.load(parser.model)) And I ran this command： python visualize.py --dataset coco --coco_path /data/COCO --model models/coco_resnet_50_map_0_335_state_dict.pt but I got this result:

[Lam1360]

Hey, I solve this problem! The problem lies in lib/build.sh. What I do is just to add ‘-gencode arch=compute_70,code=sm_70’ (according to my GPU device RTX2070) to the CUDA_ARCH and recompile the NMS extension. Now everything works well! Hope this help someone and sorry for my poor English.

[mahaling]

Here’s a bug I found in compiling nms… hope it is useful to someone. The build.sh file is saying “$CUDA_ARCH” as a parameter to nvcc command which was not set inside my docker environment. The actual nvcc command goes like the one in the link below. https://github.com/multimodallearning/pytorch-mask-rcnn

I replaced $CUDA_ARCH with --arch=sm_52 (I have a titanx).

I was getting a “no gpu_nms” module found error before and it was gone after compiling nms properly.

Hope this helps someone.