Getting "TypeError: forward() got multiple values for argument 'attention_mask'" when replacing pytorch_transformers with transformers

📚 Migration

Information

Model I am using (Bert):

Language I am using the model on (Japanese):

The problem arises when using:

• the official example scripts: (give details below)
• [* ] my own modified scripts: (give details below)

The tasks I am working on is:

• an official GLUE/SQUaD task: (give the name)
• [* ] my own task or dataset: (give details below)

Details

This is the complaint from python:

/content/train_extractive.py in train_ext(args, device_id) 225 train_multi_ext(args) 226 else: –> 227 train_single_ext(args, device_id) 228 229

/content/train_extractive.py in train_single_ext(args, device_id) 267 268 trainer = build_trainer(args, device_id, model, optim) –> 269 trainer.train(train_iter_fct, args.train_steps)

/content/trainer_ext.py in train(self, train_iter_fct, train_steps, valid_iter_fct, valid_steps) 150 self._gradient_accumulation( 151 true_batchs, normalization, total_stats, –> 152 report_stats) 153 154 report_stats = self._maybe_report_training(

/content/trainer_ext.py in _gradient_accumulation(self, true_batchs, normalization, total_stats, report_stats) 393 mask_cls = batch.mask_cls 394 –> 395 sent_scores, mask = self.model(src, segs, clss, mask, mask_cls) 396 397 loss = self.loss(sent_scores, labels.float())

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs) 720 result = self._slow_forward(*input, **kwargs) 721 else: –> 722 result = self.forward(*input, **kwargs) 723 for hook in itertools.chain( 724 _global_forward_hooks.values(),

/content/model_builder.py in forward(self, src, segs, clss, mask_src, mask_cls) 176 print (type(mask_src)) 177 print (mask_src) –> 178 top_vec = self.bert(src, segs, mask_src) 179 sents_vec = top_vec[torch.arange(top_vec.size(0)).unsqueeze(1), clss] 180 sents_vec = sents_vec * mask_cls[:, :, None].float()

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs) 720 result = self._slow_forward(*input, **kwargs) 721 else: –> 722 result = self.forward(*input, **kwargs) 723 for hook in itertools.chain( 724 _global_forward_hooks.values(),

/content/model_builder.py in forward(self, x, segs, mask) 126 def forward(self, x, segs, mask): 127 if(self.finetune): –> 128 top_vec, _ = self.model(x, segs, attention_mask=mask) 129 else: 130 self.eval()

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs) 720 result = self._slow_forward(*input, **kwargs) 721 else: –> 722 result = self.forward(*input, **kwargs) 723 for hook in itertools.chain( 724 _global_forward_hooks.values(),

TypeError: forward() got multiple values for argument ‘attention_mask’

---

I get the above complaint after replacing pytorch-transfomers with transformers.

from pytorch_transformers import BertModel -> from transformers import BertForMaskedLM

I have to make this change because I am importing the Japanese model when the original code calling BertModel only caters to English model

Environment info

python: can’t open file ‘transformers-cli’: [Errno 2] No such file or directory

• transformers version:
• Platform: ubuntu colab
• Python version: 3.6.9
• PyTorch version (GPU?): 1.6.0+cu101
• Tensorflow version (GPU?): 2.3
• Using GPU in script?: yes
• Using distributed or parallel set-up in script?:

• pytorch-transformers or pytorch-pretrained-bert version (or branch): pytorch-transformers

Checklist

• [ *] I have read the migration guide in the readme. (pytorch-transformers; pytorch-pretrained-bert)
• [ *] I checked if a related official extension example runs on my machine.

[LysandreJik]

Instead of simply removing the _ value, which will not unpack the tuple anymore, you can get the first value of the tuple (which has a single value in your case):

top_vec = self.model(x, segs, attention_mask=mask)[0]

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.