question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Lightgbm model with ONNX backend is too slow in comparison to one with torch backend

See original GitHub issue

The notebook LGBM-ONNX-example (URL) can be improved in several ways:

  1. Add handy ‘pip install’ commands cell similar to other notebooks (for improving experience in Google Colab)
  2. The example has high memory usage and also takes longer to run than the lightgbm’s pytorch backend example.
  • The memory usage is higher than 16 GB and when run in Colab, etc, the kernel is killed.
  • Similarly, the execution time also increases rapidly as we increase the number of samples.
  • The above observations may be like that by design or may indicate a memory leak or inefficient implementations. Or is it some other factor? Any ideas welcome here. 😃
  1. Fix typos in some of the cells (innitial_types vs input_types) so the users can execute the example.

Till we investigate and address the point 2, I propose that we reduce the number of samples in the example provided in the notebook so that it can be run in most systems that our users may use.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
interesaaatcommented, Jul 27, 2020

I did some profiling, and it looks that the problem is that the default opset (9) used to export into ONNX produces bad models. With opset 11 the performance are quite close to PyTorch (for GPU at least). Issue #211 should fix the problem.

1reaction
KranthiGVcommented, Jul 17, 2020

Yes, as you said, the problem seems to be in onnx converter. Thanks for the pointers here. I will do the comparison first and we can then file a bug appropriately.

Read more comments on GitHub >

github_iconTop Results From Across the Web

ONNX Runtime much slower than PyTorch (2-3x slower) #12880
So in this case with dynamic inputs, it leads to the Torch model appearing to run faster. I wrote an article with detailed...
Read more >
hummingbird-ml/community - Gitter
Hello, I have a genuine question about backends. I understood hummingbird basically transforms any model/framework to tensors and currently torch is the goto ......
Read more >
Operationalizing PyTorch Models Using ONNX and ... - NVIDIA
ONNX : an open and interoperable format for ML models ... Internally, there are two ways to convert PyTorch model to Torch IR...
Read more >
An empirical approach to speedup your BERT inference with ...
Inference takes a relatively long time compared to more modest models and it may be too slow to achieve the throughput you need....
Read more >
A Javascript library to run ONNX models in browsers ... - W3C
Based on our evaluation, compared to JavaScript, WebAssembly can improve the performance by over 11 times. We have enabled WebAssembly as one CPU...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found