It should be possible to set retention.bytes or retention.ms when creating a topic

Hi @mbogoevici,

Currently there is no way to specify topic config to use when creating new topic. I think it would be awesome to have an ability to say something like…

spring.cloud.stream.kafka.topics.foo.retention.ms = 604800000
spring.cloud.stream.kafka.topics.foo.retention.bytes = 200000

…to indicate retention limits that should be applied by broker admin utility when creating new topic foo.

It would be nice to support other topic config options as well in the future, but retention limits would be nice starting point IMHO as this is probably the most popular config option applied to new topics.

I can prepare a PR with such addition, but I just wanted to verify that this is something you find useful.

Thanks!

[hekonsek]

OK. I will create a pull request and link it here.

Cheers!

[sobychacko]

Topic level properties can be passed through the topic provisioner to admin using the binding level properties. If you have a binding named input and if its a consumer binding, then this is how you may use it. spring.cloud.stream.kafka.bindings.input.consumer.topic.

replicationFactor abnd replicaAssignments are supported by default. Any other properties need to use the following format. spring.cloud.stream.kafka.bindings.input.consumer.topic.properties.retention.ms, spring.cloud.stream.kafka.bindings.input.consumer.topic.properties.max.message.bytes etc.

See this class for more details.

If you have a producer binding named output, then replace consumer with producer. For e.g. spring.cloud.stream.kafka.bindings.output.producer.topic.

If you are using it as part of the Kafka Streams binder, then use the prefix spring.cloud.stream.kafka.streams.bindings...