Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

"CUDA out of memory." at training.

See original GitHub issue

Hi! I try to train with this command.(At my windows PC with RTX2070)

F:\Users\sounansu\Anaconda3\FCHarDNet>\python train.py --config configs\hardnet.yml
.....
RuntimeError: CUDA out of memory. Tried to allocate 40.00 MiB (GPU 0; 8.00 GiB total capacity; 5.98 GiB already allocated; 24.97 MiB free; 30.09 MiB cached)

Please teach me how to modify hardnet.yml!

Issue Analytics

State:
Created 4 years ago
Comments:8 (2 by maintainers)

Top GitHub Comments

1reaction

sounansucommented, Dec 2, 2019

Thank you another advice.

So. I modified batch size as below.

-    batch_size: 16
+    batch_size: 4

Validation values are

Total Frame Rate = 34.36 fps
Overall Acc:     0.9550504199267023
Mean Acc :       0.8234178483937832
FreqW Acc :      0.9176348176979007
Mean IoU :       0.728196572228794
0 0.9797917075564525
1 0.8329137830933465
2 0.9146849932283128
3 0.5196456033824945
4 0.5477201100625065
5 0.6087271166747225
6 0.6399322002349301
7 0.7507402719029521
8 0.9204021515861227
9 0.6014830162332917
10 0.9410106909056067
11 0.7900416534638349
12 0.5772749706560611
13 0.9361805056874339
14 0.6014725882823072
15 0.7831162906176867
16 0.6354351352905586
17 0.5277508688827295
18 0.7274112146057334
It is little better mIoU than older!
Thank you!

0reactions

18022443868commented, Mar 6, 2020

1024,2048

Please check out your version of Windows pytorch and other package versions

Top Results From Across the Web

CUDA out of memory during training - PyTorch Forums

Hello, I am pretty new to machine learning and I am facing an issue I cannot solve by myself. I took this code...

Resolving CUDA Being Out of Memory With Gradient ...

Implementing gradient accumulation and automatic mixed precision to solve CUDA out of memory issue when training big deep learning models which requires ...

Runtime error: CUDA out of memory by the end of training and ...

The problem is your loss_train list, which stores all losses from the beginning of your experiment. If the losses you put in were...

Solving "CUDA out of memory" Error - Kaggle

Solving "CUDA out of memory" Error · 1) Use this code to see memory usage (it requires internet to install package): · 2)...

Cuda out of memory error - Intermediate - Hugging Face Forums

I encounter the below error when I finetune my dataset on mbart. RuntimeError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU...