question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Crash with fp16 in pytorch 1.11

See original GitHub issue

Crash due to new signature of autocast.__exit__() in pytorch 1.11

See below, now args are required: https://github.com/pytorch/pytorch/blob/0b1f3bd158bec2cf4ab6bfde573d691c65b0742d/torch/autocast_mode.py#L201

Error:

Traceback (most recent call last):
 File "/Midgard/home/mrabadan/anaconda3/envs/multitask/lib/python3.8/contextlib.py", line 120, in __exit__
    next(self.gen)
  File "/Midgard/home/mrabadan/anaconda3/envs/multitask/lib/python3.8/site-packages/accelerate/accelerator.py", line 573, in autocast
    autocast_context.__exit__()
TypeError: __exit__() missing 3 required positional arguments: 'exc_type', 'exc_val', and 'exc_tb'

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
sguggercommented, Mar 16, 2022

Thanks a lot, I understand the problem now! Will fix this today.

1reaction
sguggercommented, Mar 15, 2022

Like I said, I have no problem with this, so I need a clear reproducer to be able to help.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Torch scripted fp16 model shuts down Kaggle notebook when ...
SOLVED: It was my local pytorch 1.12 which appears to be incompatible with Kaggle's 1.11 when it comes to jit. Hi everyone, I...
Read more >
PyTorch 1.11.0 Now Available - Exxact Corporation
PyTorch just released version 1.11 with TorchData, functorch, Distributed Data Parallel (DDP) static graph optimizations, and more.
Read more >
TensorRT 8.4.1 Release Notes - NVIDIA Documentation Center
TensorRT in FP16 mode does not perform cast operations correctly when only ... Certain spatial dimensions may have caused crashes during DLA ...
Read more >
Full system crash when using PyTorch
It's possible that the four PyTorch code bases just happen to be able to saturate the hardware better, thus causing the crash, for...
Read more >
Automatic Mixed Precision — PyTorch Tutorials 1.12.1+cu102 ...
Your network may be GPU compute bound (lots of matmuls/convolutions) but your GPU does not have Tensor Cores. In this case a reduced...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found