question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Using :memory: SQLite engine causes DB to be wiped repeatedly

See original GitHub issue

It appears that knex uses a connection pool with an idle timeout when connecting to SQLite in :memory: mode. This causes the database contents to be dropped after ~30s of inactivity (when the connection is closed), making it impossible to use that engine.

This is a particularly bad gotcha because a test suite, or non-interactive usage with something hammering the DB, will work fine.

Issue Analytics

  • State:closed
  • Created 7 years ago
  • Reactions:2
  • Comments:11 (5 by maintainers)

github_iconTop GitHub Comments

7reactions
zxxccommented, Jan 8, 2019

Faced same issue but i need pool size more than 1 So my solution is to use shared memory database

const knex = require('knex')({
  client: 'sqlite3',
  connection: { 
    filename: "file:memDb1?mode=memory&cache=shared", 
  },
  useNullAsDefault: true,
  pool: {
    min: 1,
    max: 10,
    disposeTimeout: 360000 * 1000,
    idleTimeoutMillis: 360000 * 1000
  }
});

If two or more distinct but shareable in-memory databases are needed in a single process, then the mode=memory query parameter can be used with a URI filename to create a named in-memory database: rc = sqlite3_open(“file:memdb1?mode=memory&cache=shared”, &db);

1reaction
elhigucommented, Sep 19, 2018

This could work as a workaround which sets timeouts to 100 hours.

knex = require('knex')({
  client: 'sqlite3', 
  connection: ':memory:', 
  pool: { 
    min: 1, 
    max: 1, 
    disposeTimeout: 360000*1000, 
    idleTimeoutMillis: 360000*1000 
}});
Read more comments on GitHub >

github_iconTop Results From Across the Web

File Locking And Concurrency In SQLite Version 3
Filesystem corruption following a power failure might cause the journal to be renamed or deleted.
Read more >
What an in-memory database is and how it persists data ...
It means that each time you query a database or update data in a database, you only access the main memory. So, there's...
Read more >
SQLAlchemy memory leak when instrumented objects not ...
Repeated insertions into sqlite database via sqlalchemy causing memory leak? 2 · AttributeError: module 'sqlalchemy.dialects' has no attribute ' ...
Read more >
SQLite - Quick Guide - Tutorialspoint
If the database is an in-memory or temporary database, the database will be destroyed and the contents will be lost. Syntax. Following is...
Read more >
Berkeley DB FAQ - Oracle
What are the differences between using SQLite and Berkeley DB? ... The Berkeley DB environment keeps memory for a fixed number of lockers,...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found