Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Indexing error in RPNLossComputation.prepare_targets

See original GitHub issue

🐛 Bug

File "/maskrcnn-benchmark/maskrcnn_benchmark/modeling/rpn/rpn.py", line 100, in forward
    return self._forward_train(anchors, objectness, rpn_box_regression, targets)
  File "/maskrcnn-benchmark/maskrcnn_benchmark/modeling/rpn/rpn.py", line 119, in _forward_train
    anchors, objectness, rpn_box_regression, targets
  File "/maskrcnn-benchmark/maskrcnn_benchmark/modeling/rpn/loss.py", line 91, in __call__
    labels, regression_targets = self.prepare_targets(anchors, targets)
  File "/maskrcnn-benchmark/maskrcnn_benchmark/modeling/rpn/loss.py", line 62, in prepare_targets
    labels_per_image[~anchors_per_image.get_field("visibility")] = -1
RuntimeError: The shape of the mask [5118] at index 0 does not match the shape of the indexed tensor [0] at index 0

I am getting this error and I have no idea why… At some point in RPNLossComputation.prepare_targets, matched_targets is a BoxList with num_boxes=0, resulting in an empty tensor in labels_per_image and subsequently in a indexing error down the line.

https://github.com/facebookresearch/maskrcnn-benchmark/blob/ced10f205d9f1b7b8b1d7968e1c9910e020a0165/maskrcnn_benchmark/modeling/rpn/loss.py#L50-L62

Environment

Docker image

PyTorch version: 1.0.0.dev20181119
Is debug build: No
CUDA used to build PyTorch: 9.0.176

OS: Ubuntu 16.04.4 LTS
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
CMake version: version 3.5.1

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 9.0.176
GPU models and configuration: 
GPU 0: Tesla V100-DGXS-16GB
GPU 1: Tesla V100-DGXS-16GB
GPU 2: Tesla V100-DGXS-16GB
GPU 3: Tesla V100-DGXS-16GB

Nvidia driver version: 384.145
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.1.2
/usr/lib/x86_64-linux-gnu/libcudnn_static_v7.a

Issue Analytics

State:
Created 5 years ago
Comments:7 (7 by maintainers)

Top GitHub Comments

1reaction

fmassacommented, Dec 4, 2018

The error points down to the fact that there is not GT for one image that has been fed, that is kind of clear from the error message and there is BTW and open PR to improve this error message in https://github.com/facebookresearch/maskrcnn-benchmark/pull/37

If you are adding random crops in your transforms, that is probably the culprit.

0reactions

fmassacommented, Dec 4, 2018

We don’t add random crops by default (this option is not even present in the current codebase actually)