Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
ONNX export of decoder/decoder with past has unwanted output nodes
See original GitHub issueSystem Info
Transformers commit: 3ee958207a9830da6afcf2c20772ef9a2159ca30
Optimum commit: 9c71dd52f71f397bd97da2228691a3c7049f4452
Who can help?
No response
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, …) - My own task or dataset (give details below)
Reproduction
python3 -m optimum.exporters.onnx --model t5-small --task seq2seq-lm-with-past --for-ort t5_onnx
Expected behavior
No unwanted output node
Issue Analytics
- State:
- Created 9 months ago
- Comments:5 (5 by maintainers)
Top Results From Across the Web
Recommended way of exporting encoder-decoder model to ...
I am looking for a way to export an encoder-decoder to ONNX to run inference. ... The output of the model was successful....
Read more >Exporting transformers models - Hugging Face
Namely, the ability to handle the past key values for decoder models is currently in the works. Converting an ONNX model using the...
Read more >NeMo ASR collection API - NVIDIA Documentation Center
Encoder decoder CTC-based models with Byte Pair Encoding. ... This forward is used when we need to export the model to ONNX format....
Read more >Best Practices for Neural Network Exports to ONNX
Our experience shows that is easier to export PyTorch models. If possible, choose a PyTorch source and convert it using the built-in torch.onnx...
Read more >Neural machine translation with attention | Text - TensorFlow
This tutorial: An encoder/decoder connected by attention. ... The model only has access to its previous output through the RNN state.
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
cc @JingyaHuang I’ll look into it
yes exactly Mohit!
Actually it is always returned, reference: https://github.com/huggingface/transformers/blob/7f99861218babf897c7d0d6051b43d65962671c0/src/transformers/models/t5/modeling_t5.py#L1708