Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

CUDA out of memory error

See original GitHub issue

I am getting the following error on both my minival and test-std submissions:

RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 15.75 GiB total capacity; 146.35 MiB already allocated; 22.00 MiB free; 11.65 MiB cached)

Just to give context, my local runs take up < 4GB memory for evaluation. The above error also seems to suggest that only 146.35 + 22 + 11.65 MiB memory was available to begin with. Is it possible that multiple users are assigned to the same GPUs?

Issue Analytics

State:
Created 3 years ago
Reactions:2
Comments:6 (2 by maintainers)

Top GitHub Comments

2reactions

dhruvbatracommented, May 30, 2020

We are looking into it and will get back to you. This isn’t an issue with Habitat, but EvalAI backend. I’m working with that team.

1reaction

srama2512commented, May 30, 2020

I just verified that new submissions are going through. I’ll close this issue now. Thank you again for the quick response!

Top Results From Across the Web

"RuntimeError: CUDA error: out of memory" - Stack Overflow

The error occurs because you ran out of memory on your GPU. One way to solve it is to reduce the batch size...

Solving the “RuntimeError: CUDA Out of memory” error

Solving the “RuntimeError: CUDA Out of memory” error · Reduce the `batch_size` · Lower the Precision · Do what the error says ·...

Resolving CUDA Being Out of Memory With Gradient ...

Implementing gradient accumulation and automatic mixed precision to solve CUDA out of memory issue when training big deep learning models which requires ...

Solving "CUDA out of memory" Error - Kaggle

Solving "CUDA out of memory" Error · 1) Use this code to see memory usage (it requires internet to install package): · 2)...

Stable Diffusion Runtime Error: How To Fix CUDA Out Of ...

How To Fix Runtime Error: CUDA Out Of Memory In Stable Diffusion · Restarting the PC worked for some people. · Reduce the...