[question] multiple small databases or one large database?
See original GitHub issueIs it recommended to use multiple small databases or one large database?
Either way, I will be storing many (hundreds to thousands) of LiteCollection
sized from 100MB to 1GB.
Which way will get me optimal performance, and is LiteDB affected in any way by the filesize?
Finally, what is the reason for recommending using a different database for each user? I thought that I could just store users and references to their data in a single database, which will be easier than setting up a different database for each user.
Issue Analytics
- State:
- Created 7 years ago
- Comments:5 (2 by maintainers)
Top Results From Across the Web
One big database versus many small databases [closed]
If I were to have several databases, for each company one, my logical partition would be reflected and only the relevant data would...
Read more >One large database vs several smaller databases
Is this best as a single large database, or separated out into smaller databases with thematically linked tables? These tables will grow, and ......
Read more >Design discussion: one large database or multiple smaller ...
I'm doing some planning and the database in question contains a few hundred tables. It currently is spread out over a few dozen...
Read more >Many small databases or a huge one? : r/Notion
In general, with one database, you can have similar properties in one spot to manage. Then based on the topic or type of...
Read more >One Database vs. Multiple Databases
I have a database design question that I am seeking your advice. The way I see it I have two options. If you...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Hi @0xFireball,
I prefer multiple small databases. But it’s just me, not because LiteDB. “Small” files are simple to copy, deploy, backup, delete, …
LiteDB has no problems with big collections, it’s quite fast. Of course, some operation can be slower when you have big collections, like
create index
orupdate all
. It’s very difficult because you will need change a lot of pages in memory before write on disk. It’s lock you file during this process. But if you basic operation will be insert/update one/delete one/querying… no problems. LiteDB uses skip lists to index data, it’s O(log n) complexity (100.000 or 1.000.000 documets are only 1 step between)I recommended “one database per user” because I write LiteDB for that 😄. I was looking for an embedded database to write an account web application where each user had an database. Each user can encrypt your own datafile. Each user can read/write only your own data. When user unsubscribe, it’s just delete 1 file in my server. It’s all user data in a “self contained” single file. To setup was easy: each user (email) are hashed using SHA1 and it’s datafile name.
I didnt found a no-sql database for that, so, I wrote LiteDB…
Yes, you need create many subfolders to store all datafiles. You can use some like this:
Username:
user1
SHA1:b3daa77b4c04a9551b8781d03191fe098f325e67
Database:C:\Storage\b\3\b3daa77b4c04a9551b8781d03191fe098f325e67.db
It’s create 2 levels of folders based on first 2 chars from SHA1. Will have, at max, 36 folder with 36 subfolder each (1.296 folders). Using about 200 files per folder, you can have 260.000 users.
One database per user also are cool to limit disk size per user. You can create different plans with different storage sizes. Also, using per user datafile you will not have problems with write concurrency.