Examples tests improvements

There are a few things about the examples/ tests that are suboptimal:

1. They never use cuda or fp16, even if they are available.
2. The @slow decorator used in the main tests is not importable, so there are no @slow tests.
3. test_run_glue uses distilbert-case-cased. It should use a smaller model, one of the tiny family here or a new tiny model.
4. There is no test coverage for TPU.

Any help on any of these fronts would be much appreciated!

[sshleifer]

Yes!

[sshleifer]

Good idea.

1. Yes. I think the desired behavior is if torch.cuda.is_available():

• assume fp16 is available
• run the code with fp16 and cude.

Try to do that for all tests. Some will likely break. You can add a TODO to those and keep them running on CPU for now.

1b) You probably need a GPU to do this PR.

1. There are more tests than that:

$ ls examples/**/test*.py

examples/adversarial/test_hans.py
examples/summarization/bertabs/test_utils_summarization.py
examples/summarization/test_summarization_examples.py
examples/test_examples.py
examples/token-classification/test_ner_examples.py
examples/translation/t5/test_t5_examples.py