Why gradient does not scale linearly with CTCloss?

I am trying to use CTCLoss with another loss, and want coefficients of CTCLoss smaller. But when I tried different

This is the extreme cases for small CTCLoss. (CTCLoss coefficient = 0)

criterion = CTCLoss()
loss = 0*criterion(out, targets, sizes, target_sizes)
model.zero_grad()
loss.backward()
optimizer.step()

for param in model.parameters():
     if param.grad is not None:
          grad_norm += param.grad.norm().data[0]

In above example, loss is always 0. However, fairly large gradient value still exists even CTCLoss coefficient is 0.

Can anyone explain why gradient does not scale linearly with CTCLoss?

[jinserk]

nn.CTCLoss doesn’t have the softmax in it. We need to give an explicit log softmax of the input probs to it. I’ve tested but this isn’t making the difference. I guess the most difference was happened by the gradient connection in backward()

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.