TypeError: ElectraForLanguageModelingModel object argument after ** must be a mapping, not Tensor

Describe the bug When I train Electra model I’ve got an error “TypeError: ElectraForLanguageModelingModel object argument after ** must be a mapping, not Tensor”

To Reproduce

from simpletransformers.language_modeling import LanguageModelingModel
import logging


logging.basicConfig(level=logging.INFO)
transformers_logger = logging.getLogger("transformers")
transformers_logger.setLevel(logging.WARNING)

train_args = {
    "reprocess_input_data": True,
    "overwrite_output_dir": True,
    "vocab_size": 5000,
    "train_batch_size": 16,
    "eval_batch_size": 16,
}

model = LanguageModelingModel('electra', None, args=train_args, train_files="/content/drive/My Drive/dataset_contract/train.txt")

# Mixing standard ELECTRA architectures example
# model = LanguageModelingModel(
#     "electra",
#     None,
#     generator_name="google/electra-small-generator",
#     discriminator_name="google/electra-large-discriminator",
#     args=train_args,
#     train_files="wikitext-2/wiki.train.tokens",
# )

model.train_model("/content/drive/My Drive/dataset_contract/train.txt", eval_file="/content/drive/My Drive/dataset_contract/test.txt")

model.eval_model("/content/drive/My Drive/dataset_contract/test.txt")

Error:

wandb: WARNING W&B installed but not logged in.  Run `wandb login` or set the WANDB_API_KEY env variable.
INFO:simpletransformers.language_modeling.language_modeling_model: Training of None tokenizer complete. Saved to outputs/.
/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils_base.py:1321: FutureWarning: The `max_len` attribute has been deprecated and will be removed in a future version, use `model_max_length` instead.
  FutureWarning,
INFO:simpletransformers.language_modeling.language_modeling_model: Training language model from scratch
INFO:simpletransformers.language_modeling.language_modeling_utils: Creating features from dataset file at cache_dir/
100%
11961/11961 [01:47<00:00, 111.20it/s]

100%
144978/144978 [00:02<00:00, 64320.68it/s]
INFO:simpletransformers.language_modeling.language_modeling_utils: Saving features into cached file cache_dir/electra_cached_lm_126_train.txt

INFO:simpletransformers.language_modeling.language_modeling_model: Training started
Epoch 1 of 1: 0%
0/1 [00:00<?, ?it/s]
Running Epoch 0 of 1: 0%
0/9062 [00:00<?, ?it/s]


---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-1-76a0ed631cea> in <module>()
     27 # )
     28 
---> 29 model.train_model("/content/drive/My Drive/dataset_contract/train.txt", eval_file="/content/drive/My Drive/dataset_contract/test.txt")
     30 
     31 model.eval_model("/content/drive/My Drive/dataset_contract/test.txt")

1 frames
/usr/local/lib/python3.6/dist-packages/simpletransformers/language_modeling/language_modeling_model.py in train(self, train_dataset, output_dir, show_running_loss, eval_file, verbose, **kwargs)
    564                 if args.fp16:
    565                     with amp.autocast():
--> 566                         outputs = model(**inputs)
    567                         # model outputs are always tuple in pytorch-transformers (see doc)
    568                         loss = outputs[0]

TypeError: ElectraForLanguageModelingModel object argument after ** must be a mapping, not Tensor

• Google Colab
• transformers version: 3.1.0
• Platform: Linux-4.19.112±x86_64-with-Ubuntu-18.04-bionic
• Python version: 3.6.9
• PyTorch version (GPU?): 1.6.0+cu101 (True)
• Tensorflow version (GPU?): 2.3.0 (True)
• Using GPU in script?: True

[caprone]

in train_args dict set: “fp16”: False

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.