Problem in RCNN.ROI_SAMPLE_JIT=False

Thanks a lot for your fabulous work ! Your code works well when ROI_SAMPLE_JIT=True. But a problem happens in /lib/net/rcnn_net.py when ROI_SAMPLE_JIT = False, which may relate to tensor size :

  xyz_input = pts_input[..., 0:self.rcnn_input_channel].transpose(1, 2).unsqueeze(dim=3)
  xyz_feature = self.xyz_up_layer(xyz_input)
  rpn_feature = pts_input[..., self.rcnn_input_channel:].transpose(1, 2).unsqueeze(dim=3)
  merged_feature = torch.cat((xyz_feature, rpn_feature), dim=1)
  merged_feature = self.merge_down_layer(merged_feature)
  l_xyz, l_features = [xyz], [merged_feature.squeeze(dim=3)]

In my case(default as you recommend), the processed xyz_input is a [4, 512, 64, 1, 5] tensor, which can’t be processed by the sharedMLP(128,5,1,1 conv indeed): RuntimeError: Expected 4-dimensional input for 4-dimensional weight [128, 5, 1, 1], but got 5-dimensional input of size [4, 512, 64, 1, 5] instead

[wei-OHK]

No, the released code is more clear than the original code since I have re-implemented it to make it easy to understand, but they have exactly the same network. The pretrained model is trained by saving the RoIs and the features out, which are then used by training the RCNN stage. I think the main differences are that I shuffle all the RoIs when training RCNN stage (it needs 50G+ CPU memory to load all the saved features and lots of people don’t have this condition, so I re-implemented it like the released codes to benefit more peoples. ) I have trained several networks before I released the codes, and the best moderate 3D AP is about 78.4 with ROI_SAMPLE_JIT=False.

Hi shaoshuai Thank you for the nice paper and code. I faced the same problem. And I am just wondering if it is possible for us to learn more details about your implementation (e.g. how did you shuffle all the RoIs when training RCNN). Probably those details can further help us improve our models. Thanks a lot in advance!

[yinjunbo]

I’ve tried it twice that training with ROI_SAMPLE_JIT=False and exactly followed all the instructions, however, every time it reports as bellow. RuntimeError: Expected 4-dimensional input for 4-dimensional weight [128, 5, 1, 1], but got 5-dimensional input of size [4, 512, 64, 1, 5] instead. All the settings are exactly the same with the one ROI_SAMPLE_JIT=True , except for ROI_SAMPLE_JIT=False , so is there something I missed?