Using NuScenes SECOND model in demo.py

Hi there,

I’m able to test all models (including KITTI second models) with demo.py, including the nuscenes pointspillar model.

For the SECOND nuscenes model, I get this error:

Traceback (most recent call last):
  File "demo.py", line 201, in <module>
    main()
  File "demo.py", line 129, in main
    pred_dicts, _ = model.forward(data_dict)
  File "/media/pc/Data/pcdet/PCDet/pcdet/models/detectors/second_net.py", line 11, in forward
    batch_dict = cur_module(batch_dict)
  File "/home/pc/anaconda3/envs/pcdet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/media/pc/Data/pcdet/PCDet/pcdet/models/backbones_3d/spconv_backbone.py", line 237, in forward
    x = self.conv_input(input_sp_tensor)
  File "/home/pc/anaconda3/envs/pcdet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/pc/anaconda3/envs/pcdet/lib/python3.6/site-packages/spconv/modules.py", line 123, in forward
    input = module(input)
  File "/home/pc/anaconda3/envs/pcdet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/pc/anaconda3/envs/pcdet/lib/python3.6/site-packages/spconv/conv.py", line 157, in forward
    outids.shape[0])
  File "/home/pc/anaconda3/envs/pcdet/lib/python3.6/site-packages/spconv/functional.py", line 83, in forward
    return ops.indice_conv(features, filters, indice_pairs, indice_pair_num, num_activate_out, False, True)
  File "/home/pc/anaconda3/envs/pcdet/lib/python3.6/site-packages/spconv/ops.py", line 112, in indice_conv
    int(inverse), int(subm))
RuntimeError: shape '[-1, 4, 16]' is invalid for input of size 2160

Any ideas? The pp model has very poor performance on other data, so I’m hoping SECOND does a little better. If not I am thinking of retraining nuscenes models without velocity and with less classes (only KITTI ones basically), I hope that’s possibles.

Thanks!

[josy43]

How you made it run at all ? demo.py doesnt work. When i run it with a binary file from the sweep directory, I get: RuntimeError: mat1 dim 1 must match mat2 dim 0

[MartinHahner]

how to load 10 sweeps and append a timestamp to the input point clouds exactly in demo.py? Doesn’t --data_path only follow one .bin(.npy) file? Thanks!

https://github.com/open-mmlab/OpenPCDet/blob/master/pcdet/datasets/nuscenes/nuscenes_dataset.py#L122