Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Community contribution - `BetterTransformer` integration for more models!
See original GitHub issueBetterTransformer integration for more models!
BetterTransformer API provides faster inference on CPU & GPU through a simple interface!
Models can benefit from very interesting speedups using a one liner and by making sure to install the latest version of PyTorch. A complete guideline on how to convert a new model has been created on the BetterTransformer documentation!
Here is a list of models that could be potentially supported, pick one of the architecture below and let’s discuss about the conversion!
Text models 🖊️ :
- FSMT - FSMTEncoderLayer / @Sumanth077 https://github.com/huggingface/optimum/pull/494
- MobileBERT - MobileBertLayer / @raghavanone https://github.com/huggingface/optimum/pull/506
- MBart - MBartEncoderLayer + M2M100EncoderLayer / https://github.com/huggingface/optimum/pull/516 @ravenouse
- ProphetNet - ProphetNetEncoderLayer
- RemBert - RemBertLayer
- RocBert - RocBertLayer
- RoFormer - RoFormerLayer
- Tapas - TapasLayer / https://github.com/huggingface/optimum/pull/520
Vision models 📷 :
- Detr - DetrLayer
- Flava - FlavaLayer
- GLPN - GLPNLayer
- ViLT - ViLTLayer / https://github.com/huggingface/optimum/pull/508
Audio models 🔉 :
- Speech2Text - Speech2TextLayer
- NEW: Audio Speech Transformer - ASTLayer
Let us also know if you think that some architectures can be supported that we missed. Note that for encoder-decoder based models below, we expect to convert the encoder only.
Support for decoder-based models coming soon!
Issue Analytics
- State:
- Created 10 months ago
- Reactions:2
- Comments:6 (6 by maintainers)
Top Results From Across the Web
Top Related Medium Post
Top Related StackOverflow Question
Troubleshoot Live Code
Top Related Reddit Thread
Top Related Hackernoon Post
Top Related Tweet
Top Related Dev.to Post
Top Related Hashnode Post
Hey @Sumanth077 , thanks a bunch for your interest in this issue! 🚀 Would love to assist you for the integration and let’s try to make this happen! I have updated the table above, and attaching you the contribution tutorial here 😉 Would you mind forking this repo and start opening a draft pull request so that I can start guiding you there? Also please do not hesitate to ping us here for any issue you are facing for the integration 💪
Hi @Sumanth077 , I have just replied on your PR, let’s continue the discussion there 😉