Housekeeping the storage file
See original GitHub issueHello
for long-running processes that do a lot of updates, the storage file can grow very large because NeDB appends the new document state to the file after every update.
{"_id":"djNwjBIrpw6O86dj","animal":"Polycorn","wizardLevel":10}
{"_id":"djNwjBIrpw6O86dj","animal":"Polycorn","wizardLevel":13}
{"_id":"djNwjBIrpw6O86dj","animal":"Polycorn","wizardLevel":22}
{"_id":"djNwjBIrpw6O86dj","animal":"Polycorn","wizardLevel":100}
{"_id":"djNwjBIrpw6O86dj","animal":"Polycorn","wizardLevel":11}
...
{"_id":"djNwjBIrpw6O86dj","animal":"Polycorn","wizardLevel":50}
After initializing the database (i.e. after restarting the application), NeDB cleans up the storage file and only stores the latest states.
{"_id":"djNwjBIrpw6O86dj","animal":"Polycorn","wizardLevel":50}
It would be really nice if this functionality could be called by the application to clean up the file every now and then. Or set an interval so that NeDB would do this periodically either after a certain amount of updates or based on time.
Like
db.housekeep();
or
db.setHousekeepingInterval(60 * 1000); // housekeep every minute
Issue Analytics
- State:
- Created 10 years ago
- Comments:7 (3 by maintainers)
Top Results From Across the Web
Linux system housekeeping 101: Managing file storage
One of your many duties as a system administrator is the often daunting task of keeping your system's filesystems clear of clutter.
Read more >Housekeeping - IBM
Run the following housekeeping tasks regularly: Identify processing failures for command and data files; Archive and/or remove input files.
Read more >Housekeeping - MicroStream Reference Manual
It is comprised of mechanisms for cleaning up storage files, clearing unneeded cached data and recognizing deleted entities via garbage collection.
Read more >Clean Up Files Fast: Organize Your Online Storage ... - PCMag
Organize your messy files and folders with this foolproof strategy. ... It works for cleaning up your online storage, as well as tidying...
Read more >Housekeeping: Records and Files - ASCCC
Don't bother cleaning out files less than two years old. These are relatively current, and you have a high likelihood of needing to...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I see what you mean. I added two methods to the Persistence object used by the datastore:
db.persistence.setAutocompactionInterval(interval)
, interval in millisecondsdb.persistence.stopAutocompaction
They do exactly what you want and their names are pretty explicit 😃 They are still undocumented, as I’m still wondering whether to expose them directly as methods of the datastore object or not, but you can use them already.
Great stuff! This should be documented!