A question about "denominator" obtained by the function <_compute_normalizer>

Thank you for your awesome work. I am confused about the calculation process of the denominator. Why do you need to perform additions for each step in the loop? https://github.com/kmkurn/pytorch-crf/blob/master/torchcrf/__init__.py#L245

next_score = torch.logsumexp(next_score, dim=1)

I think that this operation is done at the end of the loop and does not need to be executed inside the loop. I am confused about this. Can you give me some explanation?

[kmkurn]

Yes. Instead of sum, we do logsumexp. This is because we’re working in log probability space. Let me give an example.

Suppose we want to compute r, the sum two probabilities p and q. That is, r = p + q. This is straightforward to do. However, suppose now we’re working in log probability space. That is, we don’t have p and q, but we instead have x and y where x = log p and y = log q. Also, we now want to compute z = log r from x and y. How do we do that? Note that r = p + q = exp(x) + exp(y) and thus z = log(exp(x) + exp(y)). Notice how a sum in probability space (i.e. in terms of p, q, r) becomes a log-sum-exp in log probability space (i.e. in terms of x, y, z).

Hope this helps.

[kmkurn]

You’re welcome. Glad that I could help 😃