Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

A question about "denominator" obtained by the function <_compute_normalizer>

See original GitHub issue

Thank you for your awesome work. I am confused about the calculation process of the denominator. Why do you need to perform additions for each step in the loop?

next_score = torch.logsumexp(next_score, dim=1)

I think that this operation is done at the end of the loop and does not need to be executed inside the loop. I am confused about this. Can you give me some explanation?

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

kmkurncommented, Mar 18, 2019

Yes. Instead of sum, we do logsumexp. This is because we’re working in log probability space. Let me give an example.

Suppose we want to compute r, the sum two probabilities p and q. That is, r = p + q. This is straightforward to do. However, suppose now we’re working in log probability space. That is, we don’t have p and q, but we instead have x and y where x = log p and y = log q. Also, we now want to compute z = log r from x and y. How do we do that? Note that r = p + q = exp(x) + exp(y) and thus z = log(exp(x) + exp(y)). Notice how a sum in probability space (i.e. in terms of p, q, r) becomes a log-sum-exp in log probability space (i.e. in terms of x, y, z).

Hope this helps.

kmkurncommented, Mar 18, 2019

You’re welcome. Glad that I could help 😃

Read more comments on GitHub >

github_iconTop Results From Across the Web

Rationalizing Denominators and Numerators of Radical
Step 1: Multiply numerator and denominator by a radical that will get rid of the radical in the denominator. If the radical in...
Read more >
Intro to rationalizing the denominator | Algebra (video)
Learn for free about math, art, computer programming, economics, physics, chemistry, biology, medicine, finance, history, and more. Khan Academy is a ...
Read more >
Identifying numerators and denominators | Math (video)
I just looked at a couple of the practice problems. The denominator = the number of equal parts that make one whole unit....
Read more >
How To Find a Common Denominator? Definition, Examples ...
We can obtain common denominators by multiplying both numerator (top) and denominator (bottom) by the same amount. For example, consider the addition of...
Read more >
numden - Extract numerator and denominator - MathWorks
This MATLAB function converts A to a rational form where the numerator and denominator are relatively prime polynomials with integer coefficients.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found