Issue: Adding new tokens to bert tokenizer in QA

WARNING: This issue is a replica of this other issue open by me, I ask you sorry if I have open it in the wrong place.

Hello Huggingface’s team (@sgugger , @joeddav, @LysandreJik) I have a problem with this code base notebooks/examples/question_answering.ipynb - link ENV: Google Colab - transformers Version: 4.5.0; datasets Version: 1.5.0; torch Version: 1.8.1+cu101; I am trying to add some domain tokens in the bert-base-cased tokenizer

model_checkpoint = 'bert-base-cased'
tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
list_of_domain_tokens = ["token1", "token2", "token3"]
tokenizer.add_tokens(list_of_domain_tokens)
...
...
model = AutoModelForQuestionAnswering.from_pretrained(model_checkpoint)
print(model.device)  # cpu
model.resize_token_embeddings(len(tokenizer))
trainer = Trainer(...)

Then during the trainer.fit() call it report the attached error. Can you please tell me where I’m wrong? The tokenizer output is the usual bert inputs expressed in the form of List[List[int]] eg inputs_ids and attention_mask. So I can’t figure out where the problem is with the device Input, output and indices must be on the current device

Kind Regards, Andrea

[stas00]

We will probably have to rethink the design then, since it’s not a simple “put on device if it wasn’t already” - there are multiple cases when it shouldn’t happen. For now added a hardcoded workaround: https://github.com/huggingface/transformers/pull/11322

[sgugger]

I would rather avoid adding this, as users have been used to not have to set that argument to True when not using example scripts. Can we just add the proper line in train to put the model on the device if it was not done already?

(Sorry I didn’t catch you were using do_train in the PR you added that test, I should have caught it and commented there.)