Adjust pred_scales for different input image size?

The comment https://github.com/dbolya/yolact/issues/242#issuecomment-562907739 mentions we should adjust pred_scales/set max_size if our image is not 550x550 (backbone input size), mostly to avoid upscaling.

• should this be done automatically? (detect image dimensions, or is it applied if max_size is set? )
• how do I handle non-square (640x480) images?

Thank you

[dbolya]

I’ve done the black padding before and it’s not that desirable for images with very variable image size like COCO, since you lose a lot of pixels that way. In #270 he was specifically trying to overfit onto one image, so it’s not like the network needed that many pixels to be able to classify anyway.

For non square images right now, you can try adding that black pixel border, but the better implementation that I have on the TODO list is to just have everything a fixed non-square aspect ratio. Note that I can’t change the size of the image arbitrarily while training because of the way the prototypes create masks (the features expect a consistent image size, so the size has to be fixed at the start).

As for whether the changes to the scales should be done automatically, I don’t think so. If you look at the im400 and im700 configs you can see what changes are necessary there and they’re quite simple to extrapolate your config to those changes. I don’t want to be touching the scales automatically because what scales you want depends on your dataset, since some datasets tend to have bigger objects and others tend to have smaller ones.

[dbolya]

What I understand you’re saying (?): max_size specifies backbone-size. So with max_size=640. backbone is no longer 550x550, but 640x640. Is that correct?

It is a “pre-processing” step, but the size of the input image determines the size of the backbone layers, since P2 for instance is input size // 2, P3 is input size // 4, etc. The issue with change after training is that the weights were trained expecting a certain image size, so they probably won’t work on a different image size.