Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Polyak Averaging Params_ema initiailzation
See original GitHub issueHi, I was applying the changes in your HOWTO to add Polyak averaging and there seems to be some code missing, specifically params_ema is not initialized and so the line
optimizer, params_ema = train_step(optimizer, params_ema, batch)
causes an UnboundLocalError (params_ema referenced before assignment).
Issue Analytics
- State:
- Created 4 years ago
- Comments:5 (2 by maintainers)
Top Results From Across the Web
Polyak Averaging Explained | Papers With Code
Polyak Averaging is an optimization technique that sets final parameters to an average of (recent) parameters visited in the optimization trajectory.
Read more >Lecture 6 Optimization for Deep Neural Networks - CMSC 35246
Polyak Averaging. • On Slides but for self study: Newton and Quasi Newton ... Initialize moments variables s = 0 and r =...
Read more >Ensemble Neural Network Model Weights in Keras (Polyak ...
The simplest implementation of Polyak-Ruppert averaging involves calculating the average of the weights of the models over the last few training ...
Read more >Deep Learning Book: Chapter 8 — Optimization For Training ...
Polyak Averaging : Polyak averaging consists of averaging several points in the parameter space that the optimization algorithm traverses through ...
Read more >Do you have a plan to implement polyak averaging? #4556
FYI: I implemented a naive working sample without test code. It is a Classifier-like API. ... [1] Boris T Polyak and Anatoli B...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Er, I meant a PR 😃
@marcvanzee is in the process of changing the example diff in the README and putting up our HOWTOs in a more structured way, so I’m closing this for now. Thanks @joaogui1!