question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

RuntimeError: CUDA error: device-side assert triggered while eval

See original GitHub issue

Describe the bug I trained a distilbert model for Classification. Now when I try to use the model to eval or predict I get a RuntimeError: CUDA error: device-side assert triggered To Reproduce Steps to reproduce the behavior: Train a model using the params ClassificationModel('distilbert', 'distilbert-base-uncased', num_labels=len(df_bert_train['labels'].unique()), args={'reprocess_input_data': True, 'overwrite_output_dir': True, 'max_seq_length': 64, 'train_batch_size':16, 'fp16':False, 'num_train_epochs': 10})

Desktop (please complete the following information):

  • Linux

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:17 (7 by maintainers)

github_iconTop GitHub Comments

2reactions
ghostcommented, Mar 22, 2020

@ThilinaRajapakse Hi, Just letting you know that I faced the same error ‘RuntimeError: CUDA error: device-side assert triggered’ today with the latest version(0.22.0). After trying a lot of things to overcome it, I tried a lower version of simpletransformers(0.20.0) in the end and it worked. I was trying this example: https://towardsdatascience.com/simple-transformers-introducing-the-easiest-bert-roberta-xlnet-and-xlm-library-58bf8c59b2a3

Thanks a lot for the great wrapper 😃

0reactions
tyomik-mnemoniccommented, Sep 13, 2021

Guys use CPU (it means use_cuda = False) if u faces this situation with GPU. If problem in data u can find better exeption output, and solve problem

Read more comments on GitHub >

github_iconTop Results From Across the Web

CUDA Error: Device-Side Assert Triggered: Solved | Built In
A CUDA error: device-side assert triggered is an error that's often caused when you either have inconsistency between the number of labels and ......
Read more >
RuntimeError: CUDA error: device-side assert triggered
When I freshly train the Token Classification model (DistilBertForTokenClassification) and run a prediction for a single sentence that I ...
Read more >
Pytorch 1.6.0 - RuntimeError: CUDA error: device-side assert ...
When I run this function: def evaluate(model, dataloader, calc_loss=False): ''' Helper function to get classification accuracy and ...
Read more >
CUDA error: device-side assert triggered... When train parallel ...
I am using this code: pytorch-cifar100/train.py at master · weiaicunzai/pytorch-cifar100 · GitHub I am getting this error, when I increase ...
Read more >
RuntimeError: CUDA error: device-side assert triggered when ...
RuntimeError : CUDA error: device-side assert triggered. when trying to run either of learn.lr_find(suggestions=True). or learn.validate().
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found