question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Question: MYSQL-db consumes 100% CPU

See original GitHub issue

Dear community, I run etherpad-lite on my server and experience crashes due to MYSQL consuming 100% of the CPU. Does anybody have tips for troubleshooting or tips what the cause could be?

I am not sure if this is an adequate place to post this – so feel free to close this post, if it doesn’t belong here.

Crash is caused by:

  • mysql-database spiking to 100% CPU usage and staying at 100% until reboot/crash
  • etherpad is not responding, pad cannot be edited anymore
  • eventually server crashes
  • memory use is below 40%

Crash situation:

  • crashed after 45mins of continuous writing once, after a reboot the pad still crashed again a few times
  • on a different day a different pad crashed after 1.30h of continuous writing
  • both times there were 2 authors and about 7 other people reading (though not in read-only mode)

I am not sure, what causes this sudden mysql-CPU-spikes. I can’t find any suspicious errors corresponding to the crash time in the syslogs, mysql/error.logs or via journalctl. But maybe I am not looking for the errors in the right places?

Only after the pad crashed there where some error-logs regarding charset issues, e.g.: nodejs[976]: #033[33m[2020-07-01 15:03:04.777] [WARN] client - #033[39mError: Failed assertion: mismatched apply: 21930 / 21927 -- { errorId: '... But I think that was caused by the mysql-shutdown/reboot. Nevertheless I checked the mysql-charset settings:

  • in etherpad-lite/settings.json: "dbType" : "mysql", "dbSettings" : { … "charset": "utf8mb4" },
  • etherpad_db > charset: latin1
  • table “store” inside etherpad_db > charset: utf8mb4
  • column “key” inside table “store” inside etherpad_db > charset: utf8mb4
  • column “value” inside table “store” inside etherpad_db > charset: utf8mb4

Non-Crash Info:

  • other pads work well, e.g. smaller ones with 2 authors + 1 lurker, with 2h+ of continuous writing
  • a 15min testing session didn’t show any problems, where about 15+ people were online at the same time writing and reading
  • I can’t recreate the crash with the the load tester (9 authors). Even when adding additional long text the pads work well over time. I ran the loadtest for about 30-40mins. Maybe I should try it out longer?

Etherpad-Infos:

Server-Infos:

  • @ Digital Ocean
  • 2 vCPUs
  • 2GB Memory
  • 25GB Disk
  • Ubuntu 18
  • failed2ban installed

Thanks so much for any help! I am happy to give further information. Unfortunately the pad is no longer available. It is quite unfortunate that I don’t know how to recreate the issue yet – but I thought I’d post this question anyways, in case anyone has spontaneous tips. Best regards and take care.

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

1reaction
JohnMcLearcommented, Jul 3, 2020

More importantly update Etherpad. There is a known issue with mysql in older versions.

0reactions
nejdlcommented, Aug 4, 2020

I have updated to Etherpad version 1.8.4. as you suggested. We’ve been testing the pad for a while now and under circumstances where it had failed before. The problem did not return so I can confirm that the update has successfully fixed the problem! Thanks again for your help.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Mysql showing 100% CPU usage - DBA Stack Exchange
We are facing problem of high cpu usage for mysql process (almost 100%). Here is the information related to server. Server Infos:.
Read more >
Mysql showing 100% CPU usage [closed] - Server Fault
1 Answer 1 · Your READ : WRITE ratio (61 : 39 ) gives the clue that your engine types should be changed...
Read more >
A Simple Approach to Troubleshooting High CPU in MySQL
There are many approaches to troubleshooting high CPU usage, but here we present a simple, widely-used approach that can be adapted to ...
Read more >
How to Fix MySQL High CPU Usage? - EverSQL
How to Fix MySQL High CPU Usage? · 1. Locate the root cause · 2. Fix the performance issue · 3. Make sure...
Read more >
mySQL using 100% CPU - Stack Overflow
CPU is 100%, and in user space - this is because my innoDB buffer pool size is so large all the data is...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found