Feature Request: Modification of the RackAwareGoal behavior

BackGround

Currently, cruise-control provides a RackAwareGoal to manager the partition rebalance among the cluster. This goal need to have a rack number >= to the biggest number of topic replicas.

If the rack number is lower than that, cruise-control doesn’t be able to do a rebalance of partitions.

Issue

This prerequisite cannot be meet in some cases but the functionality of rebalance is still needed and could work.

For example in this case with a topic with 1 partition and 4 replicas :

Data Center A             | Data Center B
                          |
Broker1-rack A  replica 0 | Broker2-rack B
Broker3-rack A            | Broker4-rack B replica 1
Broker5-rack A  replica 2 | Broker6-rack B
Broker7-rack A            | Broker8-rack B replica 3

With that we are secured even if the Data Center B is totally down and Broker5 is down also, but in term of partition distribution it’s not necessary to make a distinction between brokers in the same Data Center.

Proposal

Change the behavior of the rebalance regarding the RackAwareGoal by adding an optional paramter to force the minimum rack number instead of setting it with the biggest number of topic replicas.

[mkandi]

thank you. working on adding the tests and will be sending it soon

[mkandi]

@efeg and Team, we have written a new pluggable goal to satisfy this requirement and would like to contribute it back. Hope this is ok.