Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Community contribution - `BetterTransformer` integration for more models!
See original GitHub issueBetterTransformer integration for more models!
BetterTransformer API provides faster inference on CPU & GPU through a simple interface!
Models can benefit from very interesting speedups using a one liner and by making sure to install the latest version of PyTorch. A complete guideline on how to convert a new model has been created on the BetterTransformer documentation!
Here is a list of models that could be potentially supported, pick one of the architecture below and let’s discuss about the conversion!
Text models 🖊️ :
- FSMT - FSMTEncoderLayer / @Sumanth077 https://github.com/huggingface/optimum/pull/494
- MobileBERT - MobileBertLayer / @raghavanone https://github.com/huggingface/optimum/pull/506
- MBart - MBartEncoderLayer + M2M100EncoderLayer / https://github.com/huggingface/optimum/pull/516 @ravenouse
- ProphetNet - ProphetNetEncoderLayer
- RemBert - RemBertLayer / @hchings https://github.com/huggingface/optimum/pull/545
- RocBert - RocBertLayer / @shogohida https://github.com/huggingface/optimum/pull/542
- RoFormer - RoFormerLayer
- Tapas - TapasLayer / https://github.com/huggingface/optimum/pull/520
Vision models 📷 :
- Detr - DetrLayer
- Flava - FlavaLayer / https://github.com/huggingface/optimum/pull/538
- GLPN - GLPNLayer (cannot be supported)
- ViLT - ViLTLayer / https://github.com/huggingface/optimum/pull/508
Audio models 🔉 :
- Speech2Text - Speech2TextLayer
- NEW: Audio Speech Transformer - ASTLayer / @ravenouse https://github.com/huggingface/optimum/pull/548
Let us also know if you think that some architectures can be supported that we missed. Note that for encoder-decoder based models below, we expect to convert the encoder only.
Support for decoder-based models coming soon!
Issue Analytics
- State:
- Created 10 months ago
- Reactions:7
- Comments:51 (35 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
Hi @ravenouse ! From what I got, this function is a C++ binding of the transformer encoder operation that is first defined here and fully defined here as you can see, the whole transformer encoder operations (self attention + ffn) is defined in a single operation
It is not in the list because
DebertaV2does not have a regular attention mechanism, so it is not possible to use it with BetterTransformer.