⚡️ 🍉 Performance, flickering, bisynchronicity - update&RFC

An update on what I’ve been working on recently, and my plans for the upcoming weeks and months. This is a request for comments, too, so please feel free to comment with your thoughts.

Performance

First of all: I’ve been working on making 🍉 really, really fast. Lazy loading of data making app launch time fast has been a selling point for WatermelonDB from day one. But some areas sucked. For example: adding massive amounts of data at once has not been very fast on iOS and Android.

I’ve made huge progress in 0.15, making sync time 5x faster on web and 23x faster on iOS + a lot of incremental improvements here and there. Android is not yet fast. More on that later.

Flickering

From the very beginning, 🍉 has had a fully asynchronous API, based on Promises and Observables.

Long story short, this is partly due to necessity — in 2017-2018 there has not been a good/easy/sanctioned way in React Native to make synchronous Native Modules (this has been a big selling feature for people coming from Realm RN which is really annoying to use with Chrome debugger because it’s designed as fully synchronous, and remote debugger is not). Partly due to a belief that since databases are heavy, harnessing the power of parallelism/multithreading (both on React Native, and on the web using web workers), we’ll be able to make our app a lot faster. And there are a few more potential powerful features that async api enables (you could make a network-based database adapter! 😱).

Buuuuuut. There’s just this thing: data fetched in React components that doesn’t come back synchronously means that the component will always render twice: first blank, then with content. This leads to a lot of flickering. Bad, ugly glitches, leading to poor UX.

The idea was that React Suspense is just around the corner, and it will make asynchronous data fetching and rendering really simple and awesome, and in the meantime we can use prefetching to make sure we load all necessary data ahead of time so that it’s already cached by the time it’s needed.

A year or more has passed, and React Suspense is still around the corner — and while amazing, it’s not a magic bullet (more on that later).

And prefetching has not worked super great for us, because it’s a really fragile solution. And we’ve never fully documented how to do this, so I suspect most 🍉 users just deal with glitches and flicker.

Synchronicity to the rescue (or is it?)

The “simple” solution to flickering is to just avoid asynchronicity and make data come back to the component immediately.

Multithreading is great, but it’s not a silver bullet. Without a great, reliable prefetching strategy, it may cause more problems than it solves. There are two reasons for this:

1. Without prefetching, you’re ordering data only when it’s needed - and by that point… well… it’s needed now. So you’re not really getting a lot of benefit from parallelism.
2. Our experience says that databases are fast, and React/React Native/DOM are slow. So you’re adding a lot of overhead on main thread, while only moving the minority of work to a separate thread
3. Without a strategy to avoid flickering, you’re causing A LOT more rendering passes by asynchronous operation, which are expensive.

And so we’ve been experimenting with using WatermelonDB synchronously to get rid of flickering and to improve performance.

As of v0.15, I recommend using new LokiJSAdapter({ ..., useWebWorker: false, experimentalUseIncrementalIndexedDB: true }) option. It should be worse because now DB operations are blocking the main thread. But for our app, the result is MUCH better, because there are no glitches, performance is better, and memory usage is much lower!

As of v0.16.0-0 alpha version, you can use synchronous SQLite adapter on iOS only by adding { synchronous: true } experimental option to the adapter constructor. This may be removed in future release.

What about Android? I’ll explain later.

JSI Adapter

I’ve been working for a while now on rewriting the entire SQLiteAdapter for iOS and Android with a single C++ implementation based on React Native’s jsi (javascript interface). This is really challenging, and it took me many attempts to figure out how to do this. This is because jsi is not really well documented, and almost nobody outside React Native Core Team have used this directly.

You can track my progress on this effort in this PR: https://github.com/Nozbe/WatermelonDB/pull/541/files (as of writing this, an iOS playground works; Android is not yet supported - but a proof of concept of that is here: https://github.com/Nozbe/WatermelonDB/pull/490).

Here are the goals of this project:

• the adapter is going to be a lot faster than previous implementation (it skips a lot of overhead of React Native Bridge; it’s even more low-level than TurboModules)
• this should be 2-4x on iOS, and more on Android
• synchronous operation (initially; then - by default)
• single implementation for all React Native platforms
• initially, when JSI is still in flux and people use remote Chrome Debugger (this will change when React Native Fabric comes), JSI will be opt-in, with current implementations as fallback.

I’m not currently planning to support synchronous SQLiteAdapter on Android before it’s replaced with the JSI implementation.

The bisynchronous future

So opt-in synchronicity is an important goal for now because we want to avoid flickering, and it just seems easier and better for performance, for now.

But hold on. Asynchronicity is not going away! We don’t want 🍉 to be just synchronous. Nope!

• We want to keep the capability of making asynchronous database adapters. That allows network adapters, and adapters on platforms that don’t have synchronous native module capabilities
• React Suspense is coming. It’s not a silver bullet, no, but it does allow you to build things that appear synchronously, even though the data source is asynchronous. This solves the flickering problem. What’s better, it allows you to vary React and WatermelonDB behavior based on device speed. You want flicker-free experience on fast devices. But if the device is just too slow to render content in, say, 250ms, you do want progressive rendering for the perception of speed.
• When we have that, we can start thinking about multithreading again. If we have a better solution fo prefetching, we could realistically parallelize a non-trivial amount of database work, with performance benefits
• Another thing that’s coming is React Scheduler. We want to be able to hook into it – get data immediately (synchronously) when it’s needed now, but be able to fetch data with lower priority asynchronously — for UI-only data updates (for example, updating counters), or for pre-rendering off-screen content (list virtualization).

I’m calling it “bisynchronicity” (I just made this word up) — meaning, WatermelonDB must be able to support both synchronous and asynchronous operation.

Aaaaand back to present

So this is great for the future, but we need good UX now, hence the work on synchronous operations.

There’s only one catch: as of writing this, they’re not really synchronous, because the entire WatermelonDB API is based on Promises and async functions, and Promises, by design, can not resolve synchronously. So even if there’s no multithreading, IO, or other delays, the response is scheduled in next micro task on the runloop.

This means that react components still render twice - first with empty content, and then again once promise resolves. This is not perceivable by the user, because the micro task queue blocks browser/RN rendering (so it will render properly before painting on screen). But it has real overhead, since components go through the React machinery many times.

I’ve developed a proof of concept today to measure this overhead. You can check it out here: https://github.com/Nozbe/WatermelonDB/pull/575/files . I’ve improved interaction time of switching between views in Nozbe Teams by 10% by ensuring find, fetch, count are ACTUALLY synchronous. This is a pretty huge difference.

So to support bisynchronicity, I’m thinking about how to go about refactoring internal APIs so that they can resolve synchronously.

• Promise is always async, so it doesn’t work
• a fake BisyncPromise thenable implementation could allow synchronous resolution, but it’s just begging to be used with async/await syntax, and it’s not going to be transpired correctly, so that doesn’t work
• Observables can emit both synchronously and asynchronously, but I don’t like the idea of a 100% Rx-based API, because Observables are too broad and don’t explain their intention (will this resolve synchronously and then complete, or will this emit a number of items asynchronously?); and besides - I’m planning to get rid of Rx internally (leaving only external APIs like .observe() and .observeCount()), because profiling is telling me that Rx has a non-trivial performance overhead

And so I’m thinking of plain old callbacks, like this:

count(...args, callback: Result<number> => void): void

where:

type Result<T> = { value: T } | { error: Error }
// (Result is to be treated like a standard monad, with helper functions like `mapResult`, `mapError`, `flatMapResult`)

I don’t like this at all, because callbacks are really delicate and easy to screw up. But for now, I don’t have a better idea that would be very lightweight, simple, and allow methods to resolve both synchronously and asynchronously.

WDYT?

[henrymoulton]

Kicked off an update to the native example https://github.com/henrymoulton/WatermelonDB/tree/fix/new-example/examples/native63

[henrymoulton]

I also came across recently 2 projects that use JSI for data persistence

https://github.com/mrousavy/react-native-mmkv https://github.com/greentriangle/react-native-leveldb