Database disk image is malformed error sometimes

I use Microsoft edge(chrome-based browser), and it supports the Atomics & SharedBuffer, and sometimes I get database disk image is malformed error. However, I didn’t check if it happens when multiple tabs are opened, and I am unsure under which conditions it is happening.

Here is screenshot from sentry:

The stack trace:

Error: database disk image is malformed
  at n.handleError(/assets/VaultDb.worker.d94830ba.js:15:55024)
  at e.step(/assets/VaultDb.worker.d94830ba.js:15:50388)
  at n.exec(/assets/VaultDb.worker.d94830ba.js:15:53846)
  at Ri.sqlExec(/assets/VaultDb.worker.d94830ba.js:15:148562)
  at Ri.getRecords(/assets/VaultDb.worker.d94830ba.js:15:148929)
  at Qo.getLinksOfNoteId(/assets/VaultDb.worker.d94830ba.js:15:127812)
  at MessagePort.n(/assets/VaultDb.worker.d94830ba.js:1:1863)

[rhashimoto]

I was describing alternatives to the Web Locks API to someone, and when I got to the absurd-sql approach I happened on something suspicious that seems worth investigating as a cause for this bug. It’s in Transaction.upgradeExclusive() which upgrades the lock from SHARED to RESERVED/EXCLUSIVE:

https://github.com/jlongster/absurd-sql/blob/ade37d94c78d0156da73e7bd540696b39269d9e6/src/indexeddb/worker.js#L95-L111

This method abandons its read lock by discarding the existing “readonly” IDBTransaction and attempts the upgrade by creating a new “readwrite” IDBTransaction. It then checks that no other connection modified the database in between those operations by verifying that certain bytes in the first block are unchanged.

Let’s assume that some other connection did in fact modify the database and that first block is different. What happens?

What happens is the VFS is going to return a status code that it failed to obtain the lock. But what state is it leaving behind? The Transaction instance is going to have LOCK_TYPE.EXCLUSIVE set, it will contain a “readwrite” IDBTransaction, and it has cached the new first block data. Meanwhile, SQLite thinks the lock state is still SHARED (because the upgrade failed) and it still has the original first block data in its cache. These discrepancies make me very nervous.

For one thing, the SQLite locking model is now broken. As far as SQLite knows, it is still in SHARED mode and can continue to read from the database. But the IDBTransaction it was using to read (and counting on for consistency) is now gone. If SQLite does happen to make calls that go to a new IDBTransaction, eventually it will get very confused.

For another thing, if that Transaction instance state isn’t reset and remains active, then the next attempt to upgrade the lock will succeed because it thinks it is already in the upgraded state. I don’t know if this retry actually happens, but if it does then that opens the door to corruption.

Now whether or not this is in fact the cause of this particular bug, I don’t think this upgrade approach of dropping read-only, acquiring read-write, and checking for changes can be made to work with the SQLite locking model. This is because (1) I don’t think there is a way to preserve the SHARED lock and (2) there is no protocol to tell SQLite that not only did it not upgrade the lock, it also lost its original lock.

Anyway, I’m just posting this in case it’s helpful to someone. I don’t plan on pursuing it further myself (not currently an absurd-sql user). If upgrade-or-sustain semantics (as opposed to upgrade-or-unlock) can’t be implemented immediately (which I believe will require another Worker and SharedArrayBuffer), patching the code to allow only non-shared access with a “readwrite” IDBTransaction should be relatively easy.

[rshigg]

@quolpr thanks for the heads up, that was exactly it! If I come across this issue I’ll update here 😃