Importing transformers causes segmentation fault when setting cuda device

🐛 Bug

Information

The problem arises when using: my own modified scripts: (give details below)

To reproduce

import torch
import transformers

def main(local_rank):
    torch.cuda.set_device(local_rank)
    device = torch.device('cuda', local_rank)

if __name__ == "__main__":
    print (torch.__version__)
    print (transformers.__version__)
    print (torch.cuda.is_available())
    main(0)

Expected behavior

1.4.0
2.11.0
True
Segmentation fault (core dumped)

if commenting out import transformers, everything will be fine.

Environment info

• transformers version: 2.11.0
• Platform: linux
• Python version: 3.6.8
• PyTorch version (GPU?): 1.4.0 GPU
• Tensorflow version (GPU?): None
• Using GPU in script?: Yes
• Using distributed or parallel set-up in script?: No

[huangxiaoshuo]

@daphnei thx for pointing out this! The solution for me was to upgrade torch to 1.5.1+cu92, and downgrade transformers version to 2.6.0. Quite weird problem!

I have met the exactly same problem with you, and did you find the root cause？ is this issue same to report ‘fix segmentation fault’ #2207？ The following were my test results with different versions:

1. Segmentation fault (core dumped): torch 1.2.0 ; transformers 2.11.0；cuda 10.0
2. Segmentation fault (core dumped): torch 1.2.0 ; transformers 2.6.0；cuda 10.0
3. Segmentation fault (core dumped): torch 1.2.0 ; transformers 2.5.1；cuda 10.0
4. Success: torch 1.1.0 ; transformers 2.5.1；cuda 10.0

BTW, is there a torch release of ‘1.5.1+cuda10.0’? Thanks.

[daphnei]

EDIT: This problem is caused by the sentencepiece dependency. It goes away if I comment out all references to this dependency. This will break xlnet, xlm_roberta, marian, t5, albert, reformer, and camembert, but if you are using any of the non-sentencepiece models, this should solve your problem.