Sessionstorage is constantly growing
See original GitHub issueThe session storage seems to be constantly growing When running etherpad we see a constantly growing number of session storage values in the database.
Is there a way to clean them up? We already looked into the script, which didn’t help. https://github.com/ether/etherpad-lite/blob/develop/src/bin/deleteAllGroupSessions.js . So it seems that this sessions are no group sessions.
A standard session storage entry looks like:
| sessionstorage:----KdezcUojWiGfuzBgo1L-OnF4SbbZ | {"cookie":{"path":"/","_expires":null,"originalMaxAge":null,"httpOnly":true,"sameSite":"Lax","secure":false}} |
| sessionstorage:---0sPTO0IIvtWBpbYpccywxSImVqSf_ | {"cookie":{"path":"/","_expires":null,"originalMaxAge":null,"httpOnly":true,"sameSite":"Lax","secure":false}} |
| sessionstorage:---2pUJfjIE00yXpIkrzvHvUEOCLKI2X | {"cookie":{"path":"/","_expires":null,"originalMaxAge":null,"httpOnly":true,"sameSite":"Lax","secure":false}} |
Server (please complete the following information):
- Etherpad version: 1.8.13
- OS: Debian Buster
- Node.js version (node --version): v14.16.1
- npm version (npm --version): 6.14.12
Additional context
All session storage keys values from mariadb:
MariaDB [etherpad]> SELECT COUNT(*) FROM store WHERE `key` LIKE 'sessionstorage%';
+----------+
| COUNT(*) |
+----------+
| 16306299 |
+----------+
1 row in set (14.974 sec)
None session storage keys values from mariadb:
MariaDB [etherpad]> SELECT COUNT(*) FROM store WHERE `key` NOT LIKE 'sessionstorage%';
+----------+
| COUNT(*) |
+----------+
| 54156416 |
+----------+
1 row in set (36.645 sec)
Issue Analytics
- State:
- Created 2 years ago
- Comments:6 (4 by maintainers)
Top Results From Across the Web
Removing 1Item from a Growing Array stored in Session Storage
So I have an array stored in session storage that is constantly increasing, like adding items to a shopping cart, in this case...
Read more >Session Storage is Evil - Liferay
Session storage is not evil from a developers perspective, but they are absolutely evil from deployment and runtime perspectives. Here's a short list...
Read more >Local Storage vs Session Storage vs Cookie - XenonStack
Each browser storage method has a specific maximum data size. Both storage provide a large memory capacity. To be more specific, local Storage ......
Read more >Figuring out sessionStorage size limit of a browser - SitePoint
I was able to figure out the size of sessionStorage that is used by my app using the following line of code in...
Read more >Local Storage vs. Session Storage vs. Cookies
Session storage can store data ranging between 5mb - 10mb. The exact amount of storage capacity varies as per each browser's implementation, but...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Thanks for etherpad as software and the fast reply. Thanks for pointing me to this issue.
If I calculated this correct, those 16 million session won’t require more space than 300MB, so this is fine. But I think it would be great if they could expire automatically or by or a session garbage collector job 😃
Design idea:
maxAge
, value TBD (maybe 7 days so the user doesn’t have to keep logging in?)rolling=true
so that a session stays alive as long as the user stays connectedsaveUninitialized=false
to lower the load on the db (though it might not help in our case because I think we always modify the session object)resave=false
to lower the load on the db (though it might not help due torolling=true
)SessionStore
changes:touch()
methodreq.session.touch()
each time the server receives a socket.io ping or message from the clientA problem with the above scheme: The constant
.touch()
calls might increase the load on the db considerably. We could reduce db load by skipping a db write unless it would extend the saved lifetime by more than some threshold. (In other words: Trade accuracy for reduced load.) For example, we could setmaxAge
to 14 days and only update the db record if it would extend the lifetime by 7 days or more. That should reduce db load to one write per week per session, which is trivial.Another problem: We would need to clean up old session records after a dirty shutdown, or after upgrading from a version of Etherpad that doesn’t expire sessions. ueberdb doesn’t have cursor support so we can’t just iterate over all records that match
sessionstorage:*
. We could add cursor support to ueberdb, but that would take a lot of effort. Alternatively, with some clever key prefixing we can iterate over old sessions in small batches. Here’s one approach:instanceId
: A counter that is incremented each time Etherpad starts up.caughtUpToInstanceId
: Used to clean up session state from previous Etherpad runs. Always less than or equal toinstanceId
.`sessionstore:${instanceId}:${sid}`
(instead of the current`sessionstorage:${sid}`
) and delete records`sessionstore:${caughtUpToInstanceId}:${sid}`
through`sessionstore:${instanceId - 1}:${sid}`
.`sessionstore:${caughtUpToInstanceId}:${sid}`
through`sessionstore:${instanceId}:${sid}`
(inclusive) in parallel. Of the non-null results, use the one with the highest instance ID.To clean up legacy
`sessionstorage:${sid}`
records, we could do something like this:But it would probably be better to just have the user issue a native DB query to delete all of the legacy records.