Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

unexptected behavior of AdamP on CPU?

See original GitHub issue

Hi, I’m wondering whether AdamP optimizer may have some unexpected behavior on cpu.

I trained my models of the same parameters with Adamp on cuda and cpu. Adamp gives excellent outcomes when it is run on the cuda but, interestingly, not on the cpu. More precisely, it was not able to minimize train/val loss.

Here’s a code snippet that I used to train my model.

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5

github_iconTop GitHub Comments

bhheocommented, Jul 28, 2021

Thank you for your valuable comments 👍 I will fix it when I find a clue for the problem.

bhheocommented, Jul 28, 2021

Oh, then, it is clear that AdamP causes the problem.

I’m sorry but, it is really hard to figure out the reason for cpu issue. I have read the code multiple times. Everything looks just fine.

I’m suspicious of the built-in cosine functionF.cosine_similarity. You can try an old version of AdamP( to verify the built-in cosine function. If the old version has the same problem, I have no idea what is causing the problem.

Read more comments on GitHub >

github_iconTop Results From Across the Web

M-FAC/ at master · IST-DASLab/M-FAC
# Blocked static algorithm implementation that utilizes CPU memory. # It loads and computes with `perbatch` blocks simultaneously, which makes it quite. # ......
Read more >
Strange fluid leaking from motherboard?
It looks like a small amount of condensation spreading from the heat pipe under the top-left heatsink for your VRM's. If so, it...
Read more >
How can I reduce the noise my CPU makes, like fast ...
Specialists confirmed increased motherboard noise while mice are used to zoom/rotate objects in Blender if the Blender application window(the ...
Read more >
Damp Buildings - Damp Indoor Spaces and Health - NCBI - NIH
Almost all buildings experience excessive moisture, leaks, or flooding at some point. If dampness-related problems are to be prevented, it is essential to ......
Read more >
The detrimental effects of water on electronic devices
The presence of moisture inside polymers will induce volumetric expansion, known as hygroscopic swelling. As an example during electronic components assembly ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Post

No results found

github_iconTop Related Hashnode Post

No results found