Timeout on ProduceAsync depending on the topic

Description

Hi, When I produce a message to a Kafka topic, if I doesn’t use the “test-topic” topic (default ?) I finally get a Timeout. If I do use the test-topic" topic name everything works fine.

Is there any missing configuration on my topics to accept published message ?

How to reproduce

This is perfectly working

            var config = new ProducerConfig { BootstrapServers = "kafka:9092" };

            using (var p = new ProducerBuilder<Null, string>(config).Build())
            {
                try
                {

                    var strMessage = JsonConvert.SerializeObject(message);
                    var dr = await p.ProduceAsync("test-topic", new Message<Null, string> { Value = strMessage });  
                    Console.WriteLine($"Delivered '{dr.Value}' to '{dr.TopicPartitionOffset}'");
                }
                catch (ProduceException<Null, string> e)
                {
                    Console.WriteLine($"Delivery failed: {e.Error.Reason}");
                }
            }

This is throwing an exception after a long timeout (note I only changed the topic name to “test1” that does exists)

            var config = new ProducerConfig { BootstrapServers = "kafka:9092" };

            using (var p = new ProducerBuilder<Null, string>(config).Build())
            {
                try
                {

                    var strMessage = JsonConvert.SerializeObject(message);
                    var dr = await p.ProduceAsync("test1", new Message<Null, string> { Value = strMessage });  
                    Console.WriteLine($"Delivered '{dr.Value}' to '{dr.TopicPartitionOffset}'");
                }
                catch (ProduceException<Null, string> e)
                {
                    Console.WriteLine($"Delivery failed: {e.Error.Reason}");
                }
            }

This is the exception

My docker-compose configuration

  zookeeper:
    image: wurstmeister/zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"

  kafka:
    image: wurstmeister/kafka
    container_name: kafka
    ports:
      - "9092:9092"
    environment:
      KAFKA_ADVERTISED_HOST_NAME: kafka
      KAFKA_CREATE_TOPICS: "test1:1:1"
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

Here are the list of topics inside my kafka container

Both “topic-test” and “test1” exist

Checklist

Please provide the following information:

• A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file.

• Confluent.Kafka nuget version. v 1.4.3

• Apache Kafka version. 2.12-2.2.0 from https://microbadger.com/images/wurstmeister/kafka

• Client configuration. .net core 3.1

• Operating system. Windows 10 Pro with Docker Desktop 2.3.0.3

[mhowlett]

with the default broker settings, topics are created on the fly (but generally this isn’t good practice and you should turn it off).

a leader is the partition replica that is produced/consumed from, all other replicas (in most cases) are just there to provide redundancy.

somehow you have 1001 replicas, and no leader. something is very wrong on the broker.

I’d recommend reading some introductory material on kafka to get the fundamentals

[NicolasReyDotNet]

@mhowlett I finally rebooted my cpu, everything works fine now : topics are indeed created on the fly (fine for dev env) and all topics have a leader, the Produce operation with your lib works like a charm! Thank you