Housekeeping the storage file

Hello

for long-running processes that do a lot of updates, the storage file can grow very large because NeDB appends the new document state to the file after every update.

{"_id":"djNwjBIrpw6O86dj","animal":"Polycorn","wizardLevel":10}
{"_id":"djNwjBIrpw6O86dj","animal":"Polycorn","wizardLevel":13}
{"_id":"djNwjBIrpw6O86dj","animal":"Polycorn","wizardLevel":22}
{"_id":"djNwjBIrpw6O86dj","animal":"Polycorn","wizardLevel":100}
{"_id":"djNwjBIrpw6O86dj","animal":"Polycorn","wizardLevel":11}
...
{"_id":"djNwjBIrpw6O86dj","animal":"Polycorn","wizardLevel":50}

After initializing the database (i.e. after restarting the application), NeDB cleans up the storage file and only stores the latest states.

{"_id":"djNwjBIrpw6O86dj","animal":"Polycorn","wizardLevel":50}

It would be really nice if this functionality could be called by the application to clean up the file every now and then. Or set an interval so that NeDB would do this periodically either after a certain amount of updates or based on time.

Like

db.housekeep();

or

db.setHousekeepingInterval(60 * 1000); // housekeep every minute

[louischatriot]

I see what you mean. I added two methods to the Persistence object used by the datastore:

• db.persistence.setAutocompactionInterval(interval), interval in milliseconds
• db.persistence.stopAutocompaction

They do exactly what you want and their names are pretty explicit 😃 They are still undocumented, as I’m still wondering whether to expose them directly as methods of the datastore object or not, but you can use them already.

[alexgorbatchev]

Great stuff! This should be documented!