question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Have at least some benchmarks run against the actual DB rather than the MemoryDB

See original GitHub issue

What is wrong?

Currently, all our benchmarks run against the MemoryDB. Unfortunately, that means we are missing out to measure pretty much all optimizations that are about reducing the number of hits to read/write to the database.

In other words, if some optimizations save e.g. two hits to the DB we will barely see any difference in the benchmarks as data updates in-memory are just super fast compared to the actual cost when we hit the disk (which will be the case with something like LevelDB.

How can it be fixed

Run at least some benchmarks against LevelDB. I say some because the one argument against this, is that it will increase our CI times.

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:10 (10 by maintainers)

github_iconTop GitHub Comments

1reaction
pipermerriamcommented, Oct 16, 2018

@carver This has happened twice over the weekend, I opened a github support issue as it doesn’t seam reasonable for circle CI to be rate limited for this (and I suspect they should be able to increase their cap).

1reaction
pipermerriamcommented, Oct 4, 2018

It’s less about the actual benchmark performance and more about making sure the benchmarks work.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Benchmark series: Amazon MemoryDB and how it stands ...
It is based in part on the open source Redis platform, but adds durability and persistence. AWS positions this as targeted for customers...
Read more >
Amazon MemoryDB for Redis FAQs - AWS
Find answers to frequently asked questions for Amazon MemoryDB for Redis, a Redis-compatible, durable, in-memory database service that delivers ultra-fast ...
Read more >
Deep dive on Amazon MemoryDB for Redis - YouTube
Get started with the newest database service from AWS: Amazon MemoryDB for Redis, a Redis-compatible, durable, in-memory database service ...
Read more >
Getting Started with Amazon MemoryDB for Redis - YouTube
MemoryDB is a Redis-compatible, durable, in-memory database service ... Plus, dive deep with a demo on how to get started with MemoryDB.
Read more >
AWS On Air ft. Amazon MemoryDB | AWS Events - YouTube
Jon Fritz, Head of Product Management, In-Memory Database and Caching Services demonstrates Amazon MemoryDB for Redis is a Redis-compatible, ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found