About alignment: The Code in src/align/align_dataset_mtcnn.py does not align the face.

Hi @davidsandberg ,

I’m studying the Machine Learning on Face Recognition recently and I found that your project is a great start so I’m reading all the python codes recently.

However, I found that your code in src/align/align_dataset_mtcnn.py did not align the face, and just cropped the face by bounding_boxes.

bounding_boxes, _ = align.detect_face.detect_face(img, minsize, pnet, rnet, onet, threshold, factor)
nrof_faces = bounding_boxes.shape[0]
if nrof_faces>0:
    det = bounding_boxes[:,0:4]
    img_size = np.asarray(img.shape)[0:2]
    if nrof_faces>1:
        bounding_box_size = (det[:,2]-det[:,0])*(det[:,3]-det[:,1])
        img_center = img_size / 2
        offsets = np.vstack([ (det[:,0]+det[:,2])/2-img_center[1], (det[:,1]+det[:,3])/2-img_center[0] ])
        offset_dist_squared = np.sum(np.power(offsets,2.0),0)
        index = np.argmax(bounding_box_size-offset_dist_squared*2.0) # some extra weight on the centering
        det = det[index,:]
    det = np.squeeze(det)
    bb = np.zeros(4, dtype=np.int32)
    bb[0] = np.maximum(det[0]-args.margin/2, 0)
    bb[1] = np.maximum(det[1]-args.margin/2, 0)
    bb[2] = np.minimum(det[2]+args.margin/2, img_size[1])
    bb[3] = np.minimum(det[3]+args.margin/2, img_size[0])
    cropped = img[bb[1]:bb[3],bb[0]:bb[2],:]
    scaled = misc.imresize(cropped, (args.image_size, args.image_size), interp='bilinear')
    nrof_successfully_aligned += 1
    misc.imsave(output_filename, scaled)
    text_file.write('%s %d %d %d %d\n' % (output_filename, bb[0], bb[1], bb[2], bb[3]))
else:
    print('Unable to align "%s"' % image_path)
    text_file.write('%s\n' % (output_filename))

Thank you for the great project, I really appreciate your work.

[davidsandberg]

@pavankumarkatakam: Currently I have no plans of making a tensorflow-only version of MTCNN. @achbogga: I agree. I have tried training with some rotations as extra augmentation and it looked like that could very slightly improve the results. But it was a bit inconclusive. I haven’t tried doing some more intricate alignment of the test set only but that could be interresting to try.

[achbogga]

I have been reading the facenet paper very closely, the authors claim that the network is so strong that a tight face crop over the face region is sufficient and no need of any rotation/any other transformation. That is why the alignment code produces tight crop rather than rotating the faces. However, the performance seems to increase if the alignment is also performed. Please correct me if I am wrong @davidsandberg.