parameter `ignore_keys` of `trainer.predict` not accessible in `Trainer` or `TrainingArguments`

🚀 Feature request

The predict and evaluate methods of the Trainer class provide an excellent option of ignore_keys. Here is a small example:

trainer.predict(dataset, ignore_keys=["ner_loss", "cls_loss", "ner_logits", "cls_logits"])

@sgugger

This option is however, not accessible during the normal setup of defining TrainingArguments class nor the Trainer class so the a call to trainer.train() leads to errors during the mid-training evaluation.

Motivation

I am unable to evaluate the model metrics on the validation set during the training to see if makes sense to continue.

Your contribution

I am happy to make a PR if this is seen as a genuine problem. Like always, may be I am missing something.

[sgugger]

We could add an argument for this (like ignore_keys_for_eval) yes. Let me know if you want to tackle this!

[sgugger]

Sure! Ping me when you open a PR 😃