support MongoDB

Hi there, do you have any plan to add MongoDB support? if not, i would like to do it by myself, but seems the lib/service/README document is out-dated, so is there any updated guide?

[farwayer]

@xbill82 and kuzzle team thanks for the such detailed response! It is the same cool as your documentation. I have some thoughts (and maybe suggestions) about kuzzle database layer. I will give a full answer later.

[farwayer]

I’ll start from the abstract things. This should help to understand my point of view. And I’m sorry about my bad english 😅

Kuzzle has real nice features. Events and hooks, build-in auth and permissions, transport agnostic, pub/sub with server filters, client libraries, plugins, scaling and very good code quality can help to make awesome things.

It might give the impression that it is very easy to develop you own backend based on kuzzle. Unfortunately this is not true. Kuzzle is very egocentric. What do I mean? Kuzzle allows you to extend himself but it makes you suffer.

How can you extend some backend frameworks? It’s simple in most cases:

const server = new FantasticBackendFramework(/*config*/);

server.use(new MyTransportPlugin(/*config*/));
server.use(new MyController1(/*config*/));
server.use(new MyController2(/*config*/));

server.start();

What should you do to extend kuzzle with new controller? I didn’t try to create controller yet so steps are from documentation:

1. package.json with plugin options in pluginInfo (yes controller must be npm package)
2. controllers.js with configuration (mappings?)
3. routes.js with routes
4. controller.js with controller code
5. main plugin file (index.js?)
6. add plugin to kuzzle config

So, what if I need 10 controllers with different behaviour? So much boilerplate code.

Kuzzle looks monolithic from outside. Just like big square with “KUZZLE” from your answer. You don’t use kuzzle but kuzzle uses you code with many configs you must provide.

Main idea is Kuzzle should be more flexible for using and extending. It should not be blackbox running inside docker!

---

Ok, now lets return to databases.

You wrote ElasticSearch is not meant to be used as a so-called "master" database. But what do kuzzle keep inside ES? Users and roles. Is it important data? I think so and I don’t want to keep this data inside search engine only. I can copy it to my own database but kuzzle will still use ES for auth. Yes I know I can rewrite passport strategy. And I am not sure what to do with roles.

What is about writing my own controller to other database? Yes it’s possible. And of course I would like to get all kuzzle features. So I need to notify kuzzle about document modifications and send events to plugins. In result 99% of my controller will be same as your read and write controllers except engine calling. And if you will change some internal kitchen (rename kuzzle.notifier.notifyDocumentCreate to kuzzle.notifier.notifyDocumentCreated for example) I will need to modify my controller.

Can I use client libraries out of the box with my controller? No. write and read controllers are hardcoded inside code.

Ok, so what I need to use my own database?

• write storage controller (with complex connecting to kuzzle)
• develop storage engine
• rewrite passport strategy (to keep users inside my own database)
• rewrite client library(es)

And as result I will get 3 running databases! Is it all ok? Maybe ElasticSearch is good database but kuzzle is too hard depends on it.

---

My suggestion is to develop database abstraction layer.

First of all read and write controllers should be combined into storage controller. They perform one function to provide access to persistent storage. Some actions (like now and serverInfo) should be moved out from read controller to info or server controller.

In simple case storage controller should use some internal component to select storage backend based on collection name. In more complex cases it should has ability to select backend with dynamic function (just like permission system works). We can keep part of data in one storage, part in another storage and part in both for example.

Also it need to add new plugin type (storage for example) to provide access to different databases. Common interface for such plugins should be designed.

Also it’s need to remove internalStorage and add ability to choose any database storage. As option redis can also be storage plugin and support part of this interface.

Thank you for attention!