[Trainer] add --max_train_samples --max_val_samples --max_test_samples

As we were planning to add --max_train_samples --max_val_samples --max_test_samples to all examples https://github.com/huggingface/transformers/issues/10423, I thought is there any reason why we don’t expand the Trainer to handle that?

It surely would be useful to be able to truncate the dataset at the point of Trainer to enable quick testing.

Another plus is that the metrics can then automatically include the actual number of samples run, rather than how it is done at the moment in examples.

That way this functionality would be built-in and examples will get it for free.

TODO:

1. port --max_train_samples --max_val_samples --max_test_samples to Trainer and remove the then unneeded code in run_seq2seq.py
2. extend metrics to report the number of samples as it’s done now in:

https://github.com/huggingface/transformers/blob/aca6288ff42cebded5421020f0ff088adeb446dd/examples/seq2seq/run_seq2seq.py#L590

so that all scripts automatically get this metric reported. Most likely it should be done here:

https://github.com/huggingface/transformers/blob/aca6288ff42cebded5421020f0ff088adeb446dd/src/transformers/trainer_utils.py#L224

@sgugger

[sgugger]

Yes, I think it’s the best solution.

[sgugger]

Please don’t touch the TF examples as they have not been cleaned up and will change in the near future. And yes, none of the TF examples are currently tested.