Incorporate BEVFusion's BEV Pooling operation into BEVDet

Hello, I have been trying to replace BEVDet’s QuickCumSum operation with BEVFusion’s BEV Pooling operation. https://github.com/mit-han-lab/bevfusion/tree/main/mmdet3d/ops/bev_pool

To do so, I simply have replaced https://github.com/HuangJunJie2017/BEVDet/blob/e2f4b40dd7d267a81a0ee1c7453a1229381a6f02/mmdet3d/models/necks/view_transformer.py#L164-L169 with

x = bev_pool(x, geom_feats, B, self.nx[2], self.nx[0], self.nx[1])

Where bev_pool is BEVFusion’s bev_pool cuda operation.

However, I find that although there is significant speed up, the loss is not decreasing as expected (around 14 at end of epoch 5, while it should be around 9.5).

Looking at the papers, they seem to be equivalent pooling operations, but I was hoping for some guidance in case I missed something.

Thank you!

[Divadi]

@HuangJunJie2017 I understand; I will look into it as well.

[HuangJunJie2017]

@Divadi I start trying mixed-precision today. Just a few hours ago… Where to put @force_fp32 is a good problem. I will refer to other algorithms for some suggestions.