Cassandra: Add a cluster level lock to prevent any migration while the topology of the cluster is being modified

Following a discussion with @doanduyhai, it appears that it is an extremely bad idea to make any schema changes while the topology of the cluster is being modified (more on this here: https://www.slideshare.net/doanduyhai/cassandra-nice-use-cases-and-worst-anti-patterns-no-sqlmatters-barcelona/44).

The best approach seems to create an applicative lock (implementing CassandraCluster.TryAcquireApplicationLock) that checks the existence of a predetermined keyspace and not execute migration if this item exists.

Using this, a system administrators could create that keyspace to prevent any Evolve migration from taking place (it would be wise to wait for all deployments to complete though since some migration could potentially be taking place still, the only guarantee is that no new Evolve migration is going to start), then make change to the topology and finally deleting the keyspace to release the lock and allow deployments again.

@doanduyhai, if you have any comments/suggestions 😉

[doanduyhai]

so we don’t even need a keyspace_to_lock column but only one

Yes correct. In this case to block schema change on the whole cluster, you need to use the consistency SERIAL instead of LOCAL_SERIAL for your LWT

[lecaillon]

@Pvlerick Is this always an openned issue ?