Enhancement: Support Coercing `replication-factor` when default and Broker defines a policy

Use Case: When using the spring cloud stream binder, I would like to be able to auto-provision topics with minimal boilerplate code. When connecting to a broker that defines a policy for items like replication-factor i would like the binder to coerce the default selection (in this case 1) to the required selection (in this case 3).

Reproduction: When using Confluent Cloud as my Kafka provider, I can connect to the broker successfully by attaching my configuration set to the spring.cloud.stream.kafka.binder.configuration. However, if spring attempts to create a Topic that does not exist the following error will be thrown.

ERROR 19109 --- [           main] o.s.c.s.b.k.p.KafkaTopicProvisioner      : Failed to create topics
org.apache.kafka.common.errors.PolicyViolationException: Topic replication factor must be 3 

Preventing topic creation and ultimately application startup.

Solutions: I see that the Kafka Topic provisioner does have a feature currently that coerces the partition tolerance down if allowed by the configuration properties and notifies the user as such. Can we include a default override for policy violations such as this? For example, knowing that I set the value as 2 means that i would see an error but leaving the default as 1 would allow the system to coerce it to the broker defined policy value of 3.

Thanks for reading, I look forward to input. -Derrick

[garyrussell]

We might want to keep this open and track for 3.1; it may make sense to change the default to -1.

[garyrussell]

I’ve tested it with a 2.4 broker and setting the binder property to -1.

spring.cloud.stream.kafka.binder.replication-factor=-1

has the desired effect.

The kafka server.properties needs

default.replication.factor=2

So, we’re all set when the 2.4 broker is available.