Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
unexptected behavior of AdamP on CPU?
See original GitHub issueIssue Description
Hi, I’m wondering whether AdamP optimizer may have some unexpected behavior on cpu.
I trained my models of the same parameters with Adamp on cuda and cpu. Adamp gives excellent outcomes when it is run on the cuda but, interestingly, not on the cpu. More precisely, it was not able to minimize train/val loss.
Here’s a code snippet that I used to train my model. https://github.com/Junyoungpark/PGNN/blob/main/train_pgnn.py
Issue Analytics
- State:
- Created 2 years ago
- Comments:5
Top Results From Across the Web
M-FAC/prun.py at master · IST-DASLab/M-FAC
# Blocked static algorithm implementation that utilizes CPU memory. # It loads and computes with `perbatch` blocks simultaneously, which makes it quite. # ......
Read more >Strange fluid leaking from motherboard?
It looks like a small amount of condensation spreading from the heat pipe under the top-left heatsink for your VRM's. If so, it...
Read more >How can I reduce the noise my CPU makes, like fast ...
Specialists confirmed increased motherboard noise while mice are used to zoom/rotate objects in Blender if the Blender application window(the ...
Read more >Damp Buildings - Damp Indoor Spaces and Health - NCBI - NIH
Almost all buildings experience excessive moisture, leaks, or flooding at some point. If dampness-related problems are to be prevented, it is essential to ......
Read more >The detrimental effects of water on electronic devices
The presence of moisture inside polymers will induce volumetric expansion, known as hygroscopic swelling. As an example during electronic components assembly ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Thank you for your valuable comments 👍 I will fix it when I find a clue for the problem.
Oh, then, it is clear that
AdamPcauses the problem.I’m sorry but, it is really hard to figure out the reason for
cpuissue. I have read the code multiple times. Everything looks just fine.I’m suspicious of the built-in cosine function
F.cosine_similarity. You can try an old version ofAdamP(https://github.com/clovaai/AdamP/blob/v0.2.0/adamp/adamp.py) to verify the built-in cosine function. If the old version has the same problem, I have no idea what is causing the problem.