The pytorch example summarization/run_summarization.py do not work with MBart

Environment info

• transformers version: git+https://github.com/huggingface/transformers
• Platform:
• Python version: 3.8
• PyTorch version (GPU?): 1.10.0
• Using GPU in script?: yes
• Using GPU in script?:
• Using distributed or parallel set-up in script?:

Who can help

@sgugger @NielsRogge @Narsil @patrickvonplaten

Models:

• MBart: facebook/mbart-large-cc25
• encoder-decoder models (For example, BlenderBot, BART, Marian, Pegasus, T5, ByT5): facebook/mbart-large-cc25
• Pytorch: 1.10.0

If the model isn’t in the list, ping @LysandreJik who will redirect you to the correct contributor.

HF projects:

• datasets: different repo

Examples:

• maintained examples (not research project or legacy): summarization/run_summarization.py

Information

Model I am using (Bert, XLNet …): MBart

The problem arises when using:

• the official example scripts: (give details below)
• my own modified scripts: (give details below)

The tasks I am working on is:

• an official GLUE/SQUaD task: (give the name): summarization
• my own task or dataset: (give details below)

To reproduce

Steps to reproduce the behavior:

1. run the summarization/run_summarization.py with facebook/mbart-large-cc25 as model
2. receive the error:

INFO|tokenization_utils_base.py:888] 2021-11-10 17:36:01,873 >> Assigning ['ar_AR', 'cs_CZ', 'de_DE', 'en_XX', 'es_XX', 'et_EE', 'fi_FI', 'fr_XX', 'gu_IN', 'hi_IN', 'it_IT', 'ja_XX', 'kk_KZ', 'ko_KR', 'lt_LT', 'lv_LV', 'my_MM', 'ne_NP', 'nl_XX', 'ro_RO', 'ru_RU', 'si_LK', 'tr_TR', 'vi_VN', 'zh_CN'] to the additional_special_tokens key of the tokenizer
[INFO|modeling_utils.py:1342] 2021-11-10 17:36:02,289 >> loading weights file https://huggingface.co/facebook/mbart-large-cc25/resolve/main/pytorch_model.bin from cache at /home/super/.cache/huggingface/transformers/58963b41815ac5618d9910411e018d60a3ae7d4540a66e6cf70adf29a748ca1b.bef0d2e3352d6c4bf1213c6207738ec5ecf458de355c65b2aead6671bc612138
[INFO|modeling_utils.py:1609] 2021-11-10 17:36:07,745 >> All model checkpoint weights were used when initializing MBartForConditionalGeneration.

 

[INFO|modeling_utils.py:1617] 2021-11-10 17:36:07,745 >> All the weights of MBartForConditionalGeneration were initialized from the model checkpoint at facebook/mbart-large-cc25.
If your task is similar to the task the model of the checkpoint was trained on, you can already use MBartForConditionalGeneration for predictions without further training.
Traceback (most recent call last):
  File "src/run_summarization_bart.py", line 645, in <module>
    main()
  File "src/run_summarization_bart.py", line 371, in main
    raise ValueError("Make sure that `config.decoder_start_token_id` is correctly defined")
ValueError: Make sure that `config.decoder_start_token_id` is correctly defined

MBart is in the list of model to use with this script, but this multilingual model seams different.

Expected behavior

Train summarization with pre-trained MBart correctly.

[patil-suraj]

The run_summarization.py now supports mBART thanks to @banda-larga ! Fixed by #15125

[patil-suraj]

Sorry about the super late response.

We will need to support the language codes for mBART as we do in the run_translation.py script. Will update the script in a couple of days or feel free to open a PR @nicolalandro @banda-larga if you want, happy to help with it 😃