Sudden CPU spike to 100% and disk space decreasing fast

Description

I have “btop” always running on my second monitor… Unpinned a message in a room (from Element Web) and all of the sudden (not sure if related) server’s CPU went crazy to 100% for a minute or so, until when I restarted the ansible playbook.

In this minute my DB has grown by 1GB more or less but I lost more than 2GB of space and I don’t know where exactly. After that I ran the rust-compress utility and did a FULL VACUUM and DB is back to normal now.

Steps to reproduce

I don’t have any step to reproduce, it happened out of the blue.

The only thing I was doing at that time was removing a pinned message from a room.

Homeserver

woodpeckersnest.space

Synapse Version

1.63.1

Installation Method

Other (please mention below)

Platform

I’m running the matrix-docker-ansible playbook on a Debian 11 VPS with 2GB RAM and 1 vCPU.

Relevant log output

Jul 21 21:42:11 matrix-synapse[1303919]: 2022-07-21 19:42:11,764 - synapse.handlers.pagination - 274 - WARNING - purge_history_for_rooms_in_range-0 - [purge] purging events not possible: No event found (ts 1655840531742 => stream_ordering 155865)

Anything else that would be useful to know?

No response

[roughnecks]

Thanks for explaining, I did a VACUUM FULL with the tools provided by the ansible playbook. Hope you can improve the process 👍

[richvdh]

I’ve seen history purges cause a big explosion in state groups in the past: https://github.com/matrix-org/synapse/issues/3364#issuecomment-395263702.