Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Custom Partitioners

See original GitHub issue

this is a frequently requested feature. librdkafka now provides a built in partitioner that is compatible with the java client partitioner: murmur2_random, but there are other use cases for a custom partitioner. we plan to implement this by adding a ConfigureTopics method to the producer which will have the restriction that it must be called before the topic is first used. This method will take a config dictionary, where one of the properties that can be set is a partitioner delegate.

Issue Analytics

State:
Created 5 years ago
Reactions:5
Comments:14 (6 by maintainers)

Top GitHub Comments

3reactions

mhowlettcommented, Mar 25, 2021

have a look at the integration tests for example use: https://github.com/confluentinc/confluent-kafka-dotnet/blob/master/test/Confluent.Kafka.IntegrationTests/Tests/Producer_CustomPartitioner.cs

2reactions

rohits-devcommented, Nov 8, 2019

any news on this feature?

Top Results From Across the Web

Writing Custom Partitioner for Apache Kafka

Apache Kafka provides an interface called Partitioner. This interface has three methods that we need to override for writing custom partitioner in Kafka....

Custom Partitioning an Apache Spark DataSet

Spark provides an option to create a “custom partitioner” where one can apply the logic of data partitioning on RDDs based on custom...

Custom Partitioners for PLINQ and TPL

PLINQ and the Task Parallel Library (TPL) provide default partitioners that work transparently when you write a parallel query or ForEach loop.

Apache Kafka Foundation Course - Custom Partitioner

Custom Kafka Partitioner ... We need to create our class by implementing Partitioner Interface. Your custom partitioner class must implement three methods from ......

Custom Partitioner in Kafka: Let's Take a Quick Tour!

In this article, we discuss how to create a custom partitioner in Apache Kafka in a few, short steps.