Restrict model options in transformers examples

In transformers example https://github.com/pytorch/ignite/tree/master/examples/contrib/transformers it’s up to user to override the default model which is bert-base-uncased, a lot of models take similar inputs to BERT and similar outputs too, but models like distilbert-base-uncased, distilroberta-base, bart-base (and many other models) will not work here as they work on a bit different way regarding to the inputs and outputs of the model, check here for more info: https://huggingface.co/transformers/model_doc/distilbert.html#distilbertmodel

So we will get an error similar to this while using DistilBERT

KeyError: 'token_type_ids'

And also if we used something like XLNet that won’t work well and we will have dimensions issue, because XLNet doesn’t return a pooler_output, check here https://huggingface.co/transformers/model_doc/xlnet.html#transformers.XLNetModel

So what we should do is:

• We don’t provide the option of choosing models, and we just keep BERT, and if the user wants to change the model, he can do that manually
• Keep it that way and mention in the docs that, the models that can be used here are BERT, RoBERTa (and if there is others we mentions them).

EDIT: After deciding to go with the first option, what we need to do now is to remove model from the argument, and set config['model'] = 'bert-base-uncased' manually inside run method, and mention in the docs we are using BERT by default, and maybe leave a note about if the user wants to try another model, he should that himself.

[vfdev-5]

OK for me if you think it could be understandable by other users.

[KickItLikeShika]

Initially it looks good, what do you think @vfdev-5? if that’s OK, can you try this out please? @Ishan-Kumar2