Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

WARN [SocketServer brokerId=1] Unexpected error from /IPaddress; closing connection (org.apache.kafka.common.network.Selector)

See original GitHub issue

Hi Team, i have a 3 node kafka cluster running on kubernetes with quite a standard configuration (see below) and external node ports exposed. The cluster works as expected, the only problem i have is a bunch of log messages in broker nodes

apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    version: 2.5.0
    replicas: 3
    listeners:
      plain: {}
      tls: {}
      external:
        type: nodeport
        tls: false
    config:
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      log.message.format.version: "2.5"
    storage:
      type: jbod
      volumes:
      - id: 0
        type: persistent-claim
        size: 30Gi
        class: manual
        deleteClaim: false
  zookeeper:
    replicas: 3
    storage:
      type: persistent-claim
      size: 30Gi
      class: manual
      deleteClaim: false
  entityOperator:
    topicOperator: {}
    userOperator: {}

Here’s a sample of the messages i get:

2020-08-05 15:32:44,650 WARN [SocketServer brokerId=1] Unexpected error from /IP_ADDRESS; closing connection (org.apache.kafka.common.network.Selector) [data-plane-kafka-network-thread-1-ListenerName(EXTERNAL-9094)-PLAINTEXT-9]
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 369296128 larger than 104857600)
        at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:105)
        at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:448)
        at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:398)
        at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:678)
        at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:580)
        at org.apache.kafka.common.network.Selector.poll(Selector.java:485)
        at kafka.network.Processor.poll(SocketServer.scala:861)
        at kafka.network.Processor.run(SocketServer.scala:760)
        at java.base/java.lang.Thread.run(Thread.java:834)
2020-08-05 15:32:44,650 WARN [SocketServer brokerId=1] Unexpected error from /IP_ADDRESS; closing connection (org.apache.kafka.common.network.Selector) [data-plane-kafka-network-thread-1-ListenerName(EXTERNAL-9094)-PLAINTEXT-10]
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 369296128 larger than 104857600)
        at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:105)
        at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:448)
        at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:398)
        at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:678)
        at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:580)
        at org.apache.kafka.common.network.Selector.poll(Selector.java:485)
        at kafka.network.Processor.poll(SocketServer.scala:861)
        at kafka.network.Processor.run(SocketServer.scala:760)
        at java.base/java.lang.Thread.run(Thread.java:834)
2020-08-05 15:32:44,727 WARN [SocketServer brokerId=1] Unexpected error from /IP_ADDRESS; closing connection (org.apache.kafka.common.network.Selector) [data-plane-kafka-network-thread-1-ListenerName(EXTERNAL-9094)-PLAINTEXT-11]
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 369296128 larger than 104857600)
        at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:105)
        at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:448)
        at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:398)
        at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:678)
        at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:580)
        at org.apache.kafka.common.network.Selector.poll(Selector.java:485)
        at kafka.network.Processor.poll(SocketServer.scala:861)
        at kafka.network.Processor.run(SocketServer.scala:760)
        at java.base/java.lang.Thread.run(Thread.java:834)
2020-08-05 15:32:45,412 WARN [SocketServer brokerId=1] Unexpected error from /IP_ADDRESS; closing connection (org.apache.kafka.common.network.Selector) [data-plane-kafka-network-thread-1-ListenerName(EXTERNAL-9094)-PLAINTEXT-9]
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 369296128 larger than 104857600)
        at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:105)
        at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:448)
        at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:398)
        at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:678)
        at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:580)
        at org.apache.kafka.common.network.Selector.poll(Selector.java:485)
        at kafka.network.Processor.poll(SocketServer.scala:861)
        at kafka.network.Processor.run(SocketServer.scala:760)
        at java.base/java.lang.Thread.run(Thread.java:834)

Would you please help me figuring out what’s going on? I thing it is somehow related to external nodeport configuration or ssl but i really don’t understand what i’m doing wrong

thank you in advance