RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.cuda.IntTensor instead (while checking arguments for embedding)

🐛 Bug

  File "C:\Users\temp\Aida\aida\agents\bertbot\Bert\bert_intent_classifier_pytorch.py", line 298, in process
    logits = self.model(prediction_inputs, token_type_ids=None, attention_mask=prediction_masks)
  File "C:\Users\temp\Anaconda3\envs\fresh\lib\site-packages\torch\nn\modules\module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\temp\Anaconda3\envs\fresh\lib\site-packages\transformers\modeling_bert.py", line 897, in forward
    head_mask=head_mask)
  File "C:\Users\temp\Anaconda3\envs\fresh\lib\site-packages\torch\nn\modules\module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\temp\Anaconda3\envs\fresh\lib\site-packages\transformers\modeling_bert.py", line 624, in forward
    embedding_output = self.embeddings(input_ids, position_ids=position_ids, token_type_ids=token_type_ids)
  File "C:\Users\temp\Anaconda3\envs\fresh\lib\site-packages\torch\nn\modules\module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\temp\Anaconda3\envs\fresh\lib\site-packages\transformers\modeling_bert.py", line 167, in forward
    words_embeddings = self.word_embeddings(input_ids)
  File "C:\Users\temp\Anaconda3\envs\fresh\lib\site-packages\torch\nn\modules\module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "C:\Users\temp\Anaconda3\envs\fresh\lib\site-packages\torch\nn\modules\sparse.py", line 114, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File "C:\Users\temp\Anaconda3\envs\fresh\lib\site-packages\torch\nn\functional.py", line 1484, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.cuda.IntTensor instead (while checking arguments for embedding)

Issue

Hi everyone when I run the line:

outputs = model(input_ids = b_input_ids, attention_mask=b_input_mask, labels=b_labels)

with model defined as,

model = BertForSequenceClassification.from_pretrained("bert-base-uncased", num_labels=numlabels)

It returns the stated error. However this only happens when I am on my windows computer. When I run the exact same code with the same python version and libraries it works perfectly fine. I have the most up to date version of pytorch (1.4) and transformers installed.

Any help would be greatly appreciated

Information

Using the latest version of pytorch and transformers Model I am using (Bert, XLNet …): BertForSequenceClassification Language I am using the model on (English, Chinese …): English

[chintanckg]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Had similar issue: Young Sheldon’s solution on below stackoverflow thread worked well.

https://stackoverflow.com/questions/56360644/pytorch-runtimeerror-expected-tensor-for-argument-1-indices-to-have-scalar-t

[aryamansriram]

Having the same issue, funny thing is the whole model worked for training, but while running inference on test data the error automatically showed up