Add benchmark suite
See original GitHub issueWe should track method performance using a benchmark suite like @alimanfoo mentioned in https://github.com/pystatgen/sgkit/pull/36#issuecomment-658893949.
It would be ideal if this ran as a part of the build, but timings gathered that way are probably not very reliable. Apparently integration of ASV suites in builds is usually just to make sure they work: https://github.com/astropy/astropy/issues/6149. Presumably we should have a separate, scheduled process that runs on dedicated resources in order to benchmark accurately.
As @hammer mentions in https://github.com/related-sciences/gwas-analysis/issues/10, https://github.com/ornl-oxford/genben could also be a helpful resource (scikit-allel benchmarks).
Ideas for benchmarking:
-
sgkit.utils.max_str_len
(see https://github.com/pystatgen/sgkit/pull/314#discussion_r502590724)
Issue Analytics
- State:
- Created 3 years ago
- Reactions:1
- Comments:6 (3 by maintainers)
Top Results From Across the Web
benchmark.Suite.add JavaScript and Node.js code examples
Best JavaScript code snippets using benchmark.Suite.add(Showing top 15 results out of 351) · closure.js/add.add.on. · object_comparison.js/add.on. · test/benchmark ...
Read more >goblint/bench: The benchmark suite - GitHub
Goblint benchmark suite. This contains the suite of benchmarks we currently use to evaluate Goblint. To run the benchmarks, make sure this repository...
Read more >benchmark-suite - npm
A simple suite of benchmarking tests. Latest version: 0.1.8, last published: 5 months ago. Start using benchmark-suite in your project by ...
Read more >HEP Benchmark Suite - CERN GitLab
Automates the programmatic execution of a number of benchmarks (HS06, SPEC CPU 2017, HEP-score, etc) Formerly known as cloud-benchmark-suite.
Read more >UL Procyon Professional Benchmark Suite - UL Solutions
Flexible licensing lets you pick and choose the individual benchmarks that best meet your needs. Buy just one benchmark or add more in...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Conbench from Ursa Labs could be useful for us: https://ursalabs.org/blog/announcing-conbench. cc @arunkk09 and @LiangdeLI
Just wanna drop ann-benchmarks as an inspiration. From readme: