Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
division by zero
See original GitHub issueif gradient_accumulation_steps == 1 there is division by zero when calculating G_noise and S_noise
Issue Analytics
- State:
- Created 2 years ago
- Comments:7 (4 by maintainers)
Top Results From Across the Web
Dividing by Zero
Dividing by Zero is undefined. Dividing. To see why, let us look at what is meant by "division": Division is splitting into equal...
Read more >Division by Zero
Division by zero is the operation of taking the quotient of any number x and 0, i.e., x/0. The uniqueness of division breaks...
Read more >How to Divide by Zero
There we have it: one divided by zero equals unsigned infinity. And, by extension, so does almost any other number divided by zero...
Read more >Divide by Zero — Rules & Examples
When something other than 0 is divided by 0, the result is undefined. But when 0 is divided 0, it is called indeterminate....
Read more >Division by Zero
Hidden inconspicuously in the middle is a division by zero, and at that point the proof has stepped off the brink, making all...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I’ve found with the v1 model that per_replica_batch==1 and gradient_accumulation_steps==batch is faster than the other way around
This sounds like a bug to me then. Should be submitted to JAX/XLA?