Why do centers get updated?

Hi, I don’t quite understand why the centers could get updated in David’s implementation. Could anyone please explain the mechanism behind it? My question is:

In facenet.center_loss(…) implementation, the centers are updated by a scatter_sub OP and the resulting tensor is returned:

centers = tf.scatter_sub(centers, label, diff)
loss = tf.nn.l2_loss(features - centers_batch)
return loss, centers

However, the returned tensor(centers) is not used later in train_softmax.py:

prelogits_center_loss, _ = facenet.center_loss(prelogits, ...)

So why the centers could get updated? Thanks.

[davidsandberg]

Hi @ugtony, @JianbangZ, Thanks for spotting this! I guess the running of the center_op got dropped at some point. I added a control dependency in the center loss function on a local branch:

    with tf.control_dependencies([centers]):
        loss = tf.reduce_mean(tf.square(features - centers_batch))

I 'm currently training on casia with wd=2e-4 and cl=2e-2 and maybe the performance is slightly better after 37k iterations but it’s too early to tell. Please also note that the scaling of the center loss was changed some time ago, so the center loss factor should be multiplied by the batch size.

[ugtony]

I just finished my training with center_loss_factor=2e-2 and center_loss_alpha=0.95, the performance is pretty similar to before. Another training with center_loss_alpha=0.5 is still running, but the performance isn’t better so far. Still cannot reproduce the results of the paper. 😦