Add benchmarks
See original GitHub issueWe should add a folder with airspeed velocity (ASV) benchmarks inside scikit-learn.
ASV benchmarks done by @jeremiedbb exist in https://github.com/jeremiedbb/scikit-learn_benchmarks but they currently don’t get enough exposure (i.e. contributors are not aware about them). I think we should include them e.g. under asv_bench/
folder and remove redundant scripts from benchmarks/
folder.
I just made a PR to pandas, and it’s nice when reviewers can ask to add an ASV benchmark for a PR claiming to improve performance. For this to work they need to be inside the scikit-learn repo. Whether we run them somewhere and where is a secondary concern IMO.
For the docs we can “steal” part of the section from the pandas dev guide https://pandas.pydata.org/docs/development/contributing.html#running-the-performance-test-suite
Issue Analytics
- State:
- Created 4 years ago
- Reactions:2
- Comments:8 (8 by maintainers)
Top GitHub Comments
The benchmark suite allows to do accuracy checks also. For now I just check that some metric (depending on the model) and/or that the predicted values stay exactly the same between 2 commits.
I made the changes I wanted. Changes in asv broke a functionality I thought there was no work around but just found one so it’s fine. I opened #17026. Still missing the documentation