Fix remaining issues with python/tensorflow implemenation of MTCNN

MTCNN has been implemented using python/tensorflow and can be found here. However this implementation gives slightly different results compared to the matlab/caffe implementation from the authors.

• In the matlab/caffe implementation a call to imResample is used to down-sample image patches. For the python implementation, different implementations has been attempted and opencv:s cv2.resize(img, (sz[1], sz[0]), interpolation=cv2.INTER_AREA) seems to perform best. But the result is not identical and this seems to impact performance quite a bit.
• The probablity scores for the bounding box hypotheses differs in the two implementations. This is probably due to that the convolutions are performed slightly different.

[davidsandberg]

@jrabary The MTCNN model was originally a Caffe model which has been imported to tensorflow. And since Caffe uses a different ordering of the dimensions it required some reshaping of model inputs/outputs.

[scotthong]

Hi David,

I finally got a chance to play with MTCNN again, and the following is the code to fix the image stretching and improvement on how the margin (padding) is added to the image.

Change the align_dateset_mtcnn.py of the following lines From:

                            bb[0] = np.maximum(det[0]-args.margin/2, 0)
                            bb[1] = np.maximum(det[1]-args.margin/2, 0)
                            bb[2] = np.minimum(det[2]+args.margin/2, img_size[1])
                            bb[3] = np.minimum(det[3]+args.margin/2, img_size[0])

To:

                            # To prevent the resized image from been skewed, the bounding box needs to be
                            # adjusted to become a square. Also, the size of the margin needs to be mapped
                            # from the the target scaled image space to the size in the original image space.

                            dx = det[2] - det[0]
                            dy = det[3] - det[1]
                            margin = 0
                            if(dy >= dx):
                               margin =  dy * args.margin / (args.image_size - args.margin)
                                bb[0] = np.maximum(det[0] - (dy-dx)/2 - margin/2, 0)
                                bb[2] = np.minimum(det[2] + (dy-dx)/2 + margin/2, img_size[1])
                                bb[1] = np.maximum(det[1] - margin/2, 0)
                                bb[3] = np.minimum(det[3] + margin/2, img_size[0])
                            else:
                                margin =  dx * args.margin / (args.image_size - args.margin)
                                bb[0] = np.maximum(det[0] - margin/2, 0)
                                bb[2] = np.minimum(det[2] + margin/2, img_size[1])
                                bb[1] = np.maximum(det[1]-(dx-dy)/2 - margin/2, 0)
                                bb[3] = np.minimum(det[3] + (dx-dy)/2 + margin/2, img_size[0])

After this adjustment, the original image won’t get stretched along x or y directions (except for these clamped by min/max) .

–Scott Hong