question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

use better-sqlite3 as sqlite database driver (or provide the option)

See original GitHub issue

Problem

sqlite is a synchronous access database, e.g. only one connection can read from the database at a time. So the async api that sqlite3 provides actually does more harm than good. This is better explained by the creator of better-sqlite3 than I can:

  • node-sqlite3 uses asynchronous APIs for tasks that are either CPU-bound or serialized. That’s not only bad design, but it wastes tons of resources. It also causes mutex thrashing which has devastating effects on performance.
  • node-sqlite3 exposes low-level (C language) memory management functions. better-sqlite3 does it the JavaScript way, allowing the garbage collector to worry about memory management.
  • better-sqlite3 is simpler to use, and it provides nice utilities for some operations that are very difficult or impossible in node-sqlite3.
  • better-sqlite3 is much faster than node-sqlite3 in most cases, and just as fast in all other cases. (also here: https://github.com/JoshuaWise/better-sqlite3/issues/262)

I am building a performance-heavy local web app. My gut feeling when building apps like this is that handwritten sql statements will always be more performant than ORM generated statements. Given that, I feel that picking prisma for sqlite has an extra hurdle to get over, that the driver under the hood isnt as performant as it could be.

Suggested solution

If support for this driver feels worthwhile, there are two options. Either provide a driver option in the schema datasource:

datasource sqlite {
  provider = "sqlite"
  driver = "better-sqlite3"
  url      = "file:./dev.db"
}

or just replace the internal sqlite3 database with better-sqlite3.

Alternatives

Additional context

If sqlite3 was chosen because it is backed by mapbox and therefore has a better chance of being long-lived, it appears that sqlite3 is also maintained by a single person: https://kewde.github.io/sqlite

If this looks interesting to the contributors, I am willing to open a pr with some guidance!

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Reactions:1
  • Comments:12 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
pantharshit00commented, Jun 23, 2020

We use the Rust driver for SQLite under the hood: https://github.com/rusqlite/rusqlite . We interface with it via our quaint library: https://github.com/prisma/quaint

We can’t consume any Node.js library as a driver so for this feature request to be feasiable, there should be a Rust counter part and I also assume Rust SQLite might already be faster than better-sqlite3 so that would need some investigation.

Prisma client is powered by a Rust engine under the hood. You can find the source code of that here: https://github.com/prisma/prisma-engines. We are not using any node js driver under the hood.

0reactions
Mwnicommented, Apr 12, 2022
Read more comments on GitHub >

github_iconTop Results From Across the Web

better-sqlite3/api.md at master - GitHub
options.verbose : provide a function that gets called with every SQL string executed by the database connection (default: null ). options.
Read more >
better-sqlite3 - npm
better -sqlite3 is simpler to use, and it provides nice utilities for some operations that are very difficult or impossible in node-sqlite3 ....
Read more >
How can i use TypeORM with better-sqlite3 - Stack Overflow
1 Answer 1 · Export your existing database. · Update typeorm to 0.2.26 or higher. · Install the package better-sqlite3 . sqlite3 can...
Read more >
SQLite Adapter Options - What is RelStorage? - Read the Docs
This adapter uses the built-in sqlite3 module provided by the Python standard library. It is available on Python 2.7 (including PyPy) and Python...
Read more >
Appropriate Uses For SQLite
SQLite is not directly comparable to client/server SQL database engines such as MySQL, Oracle, PostgreSQL, or SQL Server since SQLite is ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found