Consistent speech model input names for the Seq2SeqTrainer generate function

🚀 Feature request

Could we maybe have a consistent naming convention for speech models? So far we have:

• input_features
• input_values
• input_ids

From what I can tell, these are mostly the same for the purposes of how the Seq2SeqTrainer interprets them.

Motivation

This would prevent the need for custom Seq2SeqTrainer classes and would make training more modular.

Your contribution

A change in param names would do the trick but could break a lot of code. Alternatively adding the capability to accept different key values in the generate function here would work too using a (clunky) mapping such as INPUT_MAPPING_LABELS = {"input_features": "input_ids", "input_values": "input_ids", "input_ids": "input_ids"}.

[patrickvonplaten]

Great catch @OllieBroadhurst! I’ll include a fix in https://github.com/huggingface/transformers/pull/14856 . Hope to have it merged by tomorrow

[OllieBroadhurst]

Hey there. Looks cool! One super edge case is SpeechEncoderDecoderModel.

For example, if we use a Wav2Vec2Model encoder and a XLMRobertaForCausalLM decoder, we would have ['input_ids', 'attention_mask'] when using either processor.tokenizer.model_input_names for Wav2Vec2 or tokenizer.model_input_names for XLMRoBERTa, when in fact the model requires input_values in the generate function.