the align function may Shear the face image

H = cv2.getAffineTransform(npLandmarks[npLandmarkIndices],
                                   imgDim * MINMAX_TEMPLATE[npLandmarkIndices]*scale + imgDim*(1-scale)/2)
        thumbnail = cv2.warpAffine(rgbImg, H, (imgDim, imgDim))

I got the following result:

Why not use least square？ Y = MX M = (YXt)(XXt)-1

[davidsandberg]

Yes, I think this is a known problem which has been discussed for example here. To my knowledge it has not been really solved though. Have you tried this approach? Does it produce better results?

Another approach would be to not do any transformation at all but instead just use a bounding box and let the CNN handle any rotations etc within that box. Rotations can then be seen as a kind of data augumentation instead. I have tried this approach but using MTCNN for face alignment, and when training a Inception-Resnet-v1 network on this data I can get a model with accuracy ~0.975 on LFW. Not sure how much of the performance improvement that can be attributed to not having the shearing effect though.

[davidsandberg]

I’m pretty sure I ran into the same problem but now I’m not sure how it was solved. Could have been that it was fixed when upgrading to a newer version of slim, but I’m not sure. One option would be to upgrade to TF 0.11 and see if that fixes the problem. I will do that as well as soon as I’m done with some training. If the problem persists, please file a new issue specifically for this.