Lightgbm model with ONNX backend is too slow in comparison to one with torch backend
See original GitHub issueThe notebook LGBM-ONNX-example
(URL) can be improved in several ways:
- Add handy ‘pip install’ commands cell similar to other notebooks (for improving experience in Google Colab)
- The example has high memory usage and also takes longer to run than the lightgbm’s pytorch backend example.
- The memory usage is higher than 16 GB and when run in Colab, etc, the kernel is killed.
- Similarly, the execution time also increases rapidly as we increase the number of samples.
- The above observations may be like that by design or may indicate a memory leak or inefficient implementations. Or is it some other factor? Any ideas welcome here. 😃
- Fix typos in some of the cells (
innitial_types
vsinput_types
) so the users can execute the example.
Till we investigate and address the point 2, I propose that we reduce the number of samples in the example provided in the notebook so that it can be run in most systems that our users may use.
Issue Analytics
- State:
- Created 3 years ago
- Comments:6 (3 by maintainers)
Top Results From Across the Web
ONNX Runtime much slower than PyTorch (2-3x slower) #12880
So in this case with dynamic inputs, it leads to the Torch model appearing to run faster. I wrote an article with detailed...
Read more >hummingbird-ml/community - Gitter
Hello, I have a genuine question about backends. I understood hummingbird basically transforms any model/framework to tensors and currently torch is the goto ......
Read more >Operationalizing PyTorch Models Using ONNX and ... - NVIDIA
ONNX : an open and interoperable format for ML models ... Internally, there are two ways to convert PyTorch model to Torch IR...
Read more >An empirical approach to speedup your BERT inference with ...
Inference takes a relatively long time compared to more modest models and it may be too slow to achieve the throughput you need....
Read more >A Javascript library to run ONNX models in browsers ... - W3C
Based on our evaluation, compared to JavaScript, WebAssembly can improve the performance by over 11 times. We have enabled WebAssembly as one CPU...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I did some profiling, and it looks that the problem is that the default opset (9) used to export into ONNX produces bad models. With opset 11 the performance are quite close to PyTorch (for GPU at least). Issue #211 should fix the problem.
Yes, as you said, the problem seems to be in onnx converter. Thanks for the pointers here. I will do the comparison first and we can then file a bug appropriately.