All nodes are busy but nothing is going on

Hi again! 😃 I’m running 2.2.5 in SIDECAR mode and my repair is progressing very slowly (it has repaired 19/3151 segments in the first 5 minutes and there has been no progress for the next 20 as I’m writing this issue). The event logs say All nodes are busy or have too many pending compactions for the remaining candidate segments. - but there’s no compaction going on on any node and the nodes are not busy as far as I can tell - they’re not repairing, have no write load and very light read load.

This is a fresh install of Reaper, since I’ve ran into this issue first thing after upgrading to 2.0.5 -> 2.2.5 – I recreated the backend keyspace, started a single instance to prepare the tables (on a previous attempt I believe I ran into a deadlock when all instances tried to prepare tables - every one was waiting for a migration lock), and then started all the sidecars when done.

[kzemek]

Hey @adejanovski , sorry that this has turned into a support ticket! Thank you for your help, much appreciated 😃

[adejanovski]

I’ll close this ticket as Reaper seems to be working as expected here. Feel free to keep the conversation going if you have questions or reopen if you think there’s really something wrong.