inconsistant anchor reference (may be the cause of the lower accuracy than faster-rcnn)

Hi,

Please notice you have an inconsistant reference to the order of the anchors (lines 86 and 79)

https://github.com/jwyang/fpn.pytorch/blob/23bd1d2fa09fbb9453f11625d758a61b9d600942/lib/model/rpn/rpn_fpn.py#L86

https://github.com/jwyang/fpn.pytorch/blob/23bd1d2fa09fbb9453f11625d758a61b9d600942/lib/model/rpn/rpn_fpn.py#L79

lets say k is the number of anchors

than in line 79 you are doing softmax after reshape which make anchors: 0:k-1 assosiated with proposal = false k:2k-1 assosiated with proposal = True.

and in line 86 your anchors are arranged in a different way: i mod 2 == 0 associated with proposal = false i mod 2 == 1 associated with proposal = True.

I propose to reshape the score in this way to be consistant. (You will also need to change few other things in proposalLayer for it to work)

rpn_cls_scores.append(rpn_cls_score_reshape.permute(0, 2, 3, 1).contiguous().view(batch_size, -1, 2))

[Feynman27]

Okay, I see here that the indices of the scores/probs must exactly match the indices of the bbox_deltas. I’ve changed the anchor generation minimally such that the enumeration order is (Width x Height x Anchor):

# Reshape to get a list of (x, y) and a list of (w, h)
#  Enumerate by (Width x Height x Anchor)
box_centers = np.stack([box_centers_x.transpose(1,0).reshape(-1),
                        box_centers_y.transpose(1,0).reshape(-1)], axis=1)
box_sizes = np.stack([box_widths.transpose(1,0).reshape(-1),
                     box_heights.transpose(1,0).reshape(-1)], axis=1)
# Convert to corner coordinates (x1, y1, x2, y2)
boxes = np.concatenate([box_centers - 0.5 * box_sizes,
              box_centers + 0.5 * box_sizes], axis=1)

[Feynman27]

So maybe we could match the order by doing something like:

rpn_bbox_pred = self.RPN_bbox_pred(rpn_conv1)
# reshape to match order of anchors
rpn_bbox_pred_reshape = rpn_bbox_pred.view(rpn_bbox_pred.size(0), 
                                           rpn_bbox_pred.size(1)/4, 
                                           4, 
                                           rpn_bbox_pred.size(2), 
                                           rpn_bbox_pred.size(3))
rpn_bbox_pred_reshape = rpn_bbox_pred_reshape.permute(0,1,3,4,2).contiguous().view(batch_size, -1, 4)

This would have an enumeration order of (Width x Height x Anchor)