🚀 Add built-in support for autorregressive text generation with ONNX models.

🚀 Add built-in support for autorregressive text generation with ONNX models.

After converting a autorregressive model to ONNX, it would be nice to be able to generate text with it via something like:

from transformers import OnnxTextGenerationModel, AutoTokenizer

model_path = "gpt-something.onnx"
tokenizer_name = "gpt2"

model = OnnxTextGenerationModel(model_path)

# and then

text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='tf')
output = model.generate(encoded_input)

With support to using past_key_values internally in the most efficient way.

Motivation

When trying to accelerate inference with transformers, being unable to load our ONNX model with the lib and running a model.generate method to seamlessly generate sequences and perform Beam Search is somehow frustrating. That leads us to have to rely on custom implementations - which takes time and are a lot more prone to have bugs.

[piEsposito]

Hugging Face rocks.

[harishankar-gopalan]

@philschmid Thanks for the update. I did manage to find out after some deep diving into the HuggingFace Transformers repo.