Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Set custom topic configs in NewTopic object at KafkaTopicProvisioner.class

See original GitHub issue

Currently it is impossible to pass any custom param to the topic creation method, and set the configs in the NewTopic object.

We are trying to set the message.format.version config in a topic level, when creating a topic.

Topic creation source code at KafkaTopicProvisioner class

NewTopic newTopic = new NewTopic(topicName, effectivePartitionCount,
(short) configurationProperties.getReplicationFactor());
CreateTopicsResult createTopicsResult = adminClient.createTopics(Collections.singletonList(newTopic));

It would be helpful to define a yml configuration params for topics, such as it is with the producer and consumer.

Thanks

Issue Analytics

State:
Created 6 years ago
Comments:7 (5 by maintainers)

Top GitHub Comments

2reactions

artembilancommented, Mar 6, 2018

OK. Let’s fix it on the framework level and forget my suggestion!

0reactions

doron-levicommented, Mar 6, 2018

Thanks @artembilan, can you have a look at the same time at KafkaTopicProvisioner and its AdminClient relation?

Can the AdminClient be a Bean so it might be reused in other use cases? alternately can the client be auto closed after operations (or on idle time or some other policy) ? I think I have observed it “spiting” errors to the log in some cases, it probably doing stuff in the background…

Top Results From Across the Web

Programmatically create Kafka topics using Spring Kafka

To create a Kafka topic programmatically introduce a configuration class that annotated with @Configuration : this annotation indicates that ...

Class NewTopic - Apache Kafka

Set the configuration to use on the new topic. Parameters: configs - The configuration map. Returns: This NewTopic object.

Spring Cloud Stream Kafka Binder Reference Guide

The list of custom headers that are transported by the binder. ... Binder's Kafka topic provisioner gives the highest precedence for the properties...

Kafka Topic Configurations for Confluent Platform

This topic provides configuration parameters available for Confluent Platform. The Apache Kafka® topic configuration parameters are organized by order of ...

Java Examples & Tutorials of NewTopic.configs (org.apache ...

Create a Kafka topic with the given parameters. * * @param topic The name of the topic. * @param partitions The number of...