Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Mish inplace question
See original GitHub issueHi, thanks for the great repo! What’s the reason that the x.mul_(inner) line is commented out? Does the commented line return different results? Thanks!
Issue Analytics
- State:
- Created 4 years ago
- Comments:5 (3 by maintainers)
Top Results From Across the Web
Mish — PyTorch 1.13 documentation
Mish(inplace=False)[source ]. Applies the Mish function, element-wise. Mish: A Self Regularized Non-Monotonic Neural Activation Function.
Read more >New Activation function, possible successor to ReLU? - fastai
My opinion is Mish is a more perfect activation function based on theory at least - smoother = better information propagation and thus...
Read more >2022 Pre-conference Workshop Registration: Sense of Place with ...
We'll tackle all these questions and more in this workshop intended to help poets and prose writers alike ground their writing in place....
Read more >Do Tight Labor Market Conditions Keep Core Inflation Sticky?
What's the role of tight labor markets on core inflation and overall inflation? Author: Mish. Publish date: Sep 28, 2022.
Read more >Analyzing Murphy on Mish's Deflationism | Seeking Alpha
... thinks that there are some questions on Mish's deflationist stance. ... that can be) will still be in place thus combating the...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
It’s dead code now, I should get rid of it… when I first implemented mish I though I’d be smart and make it a little more efficient with the inplace variation… however it triggered an inplace gradient error (during training)
I believe you put your training loop, or call to it under a detect_anomaly context manager… been a while since I’ve used it, has a performance impact but this sort of error should be detected quickly.