mirromaker2 can not reconcile - NullPointerException

Please use this to only for bug reports. For questions or when you need help, you can use the GitHub Discussions, our #strimzi Slack channel or out user mailing list.

Describe the bug I created a mirrormaker2 object and the operator failed to reconcile. The error logs is supplied below.

We use strimizi version 0.26.0 installed by helm. Based on this page https://strimzi.io/downloads/ it should support kafka version 3.0.0 as well however I am not sure if teh version supported.

The reason I said so because we have another mirrormaker2 object which connect to the source kafka run version 2.3.0 and it seems to work fine.

The manifest is nearly identical to the working one except different credential to connect to the source. They sync to the same target cluster.

To Reproduce Steps to reproduce the behavior:

Expected behavior

It should work - that is create a new pod, runs the replication etc … same as the working manifest.

As stated, the working manifest is identical to the config above just having different name, and source cluster details. Traget are exactly the same.

Environment (please complete the following information):

• Strimzi version: 0.26.0
• Installation method: Helm chart
• Kubernetes cluster: AWS EKS Server Version: version.Info{Major:“1”, Minor:“21+”, GitVersion:“v1.21.9-eks-0d102a7”}
• Infrastructure: EKS

YAML files and logs

manifest sample

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaMirrorMaker2
metadata:
  annotations:
  name: strimzi-kafka-to-cold-mirror
  namespace: kafka-cold
spec:
  clusters:
  - alias: cold-kafka
    authentication:
      passwordSecret:
        password: XXX
        secretName: XXX
      type: scram-sha-512
      username: XXX
    bootstrapServers: cold-kafka-bootstrap.kafka-cold.svc.cluster.local:9093
    config:
      config.storage.replication.factor: 3
      offset.storage.replication.factor: 3
      producer.max.request.size: 15728640
      ssl.cipher.suites: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
      ssl.enabled.protocols: TLSv1.2
      ssl.protocol: TLSv1.2
      status.storage.replication.factor: 3
    tls:
      trustedCertificates:
      - certificate: ca.crt
        secretName: cold-cluster-ca-cert
  - alias: strimzi-broker-kafka
    authentication:
      passwordSecret:
        password: XXX
        secretName: XXX
      type: scram-sha-512
      username: XXX
    bootstrapServers: strimzi-broker-kafka-bootstrap.kafka.svc.cluster.local:9093
    tls:
      trustedCertificates:
      - certificate: ca.crt
        secretName: cold-cluster-ca-cert
  connectCluster: cold-kafka
  livenessProbe:
    initialDelaySeconds: 60
    timeoutSeconds: 30
  logging:
    loggers:
      connect.root.logger.level: INFO
    type: inline
  metricsConfig:
    type: jmxPrometheusExporter
    valueFrom:
      configMapKeyRef:
        key: metrics-config.yml
        name: kafka-cold-kafka-jmx-exporter
  mirrors:
  - checkpointConnector:
      config:
        checkpoints.topic.replication.factor: 1
        replication.policy.class: io.strimzi.kafka.connect.mirror.IdentityReplicationPolicy
        replication.policy.separator: ""
    heartbeatConnector:
      config:
        heartbeats.topic.replication.factor: 1
    sourceCluster: strimzi-broker-kafka
    sourceConnector:
      config:
        fetch.max.bytes: 26314400
        offset-syncs.topic.replication.factor: 2
        replication.factor: 2
        replication.policy.class: io.strimzi.kafka.connect.mirror.IdentityReplicationPolicy
        replication.policy.separator: ""
        sync.topic.acls.enabled: "false"
        sync.topic.configs.enabled: "false"
    targetCluster: cold-kafka
    topicsPattern: .*
  readinessProbe:
    initialDelaySeconds: 60
    timeoutSeconds: 30
  replicas: 1
  template:
    pod:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: group
                operator: In
                values:
                - cold-broker
      tolerations:
      - effect: NoSchedule
        key: reserved
        value: cold-broker
  # version: 3.0.0

Log:

2022-05-23 05:15:04 ERROR AbstractOperator:240 - Reconciliation #335(timer) KafkaMirrorMaker2(kafka-cold/strimzi-kafka-to-cold-mirror): createOrUpdate failed java.lang.NullPointerException: null at io.strimzi.operator.common.Util.lambda$authTlsHash$10(Util.java:558) ~[io.strimzi.operator-common-0.26.0.jar:0.26.0] at io.vertx.core.impl.future.Composition.onSuccess(Composition.java:38) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.FutureBase.emitSuccess(FutureBase.java:60) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.FutureImpl.tryComplete(FutureImpl.java:211) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.Composition$1.onSuccess(Composition.java:62) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.FutureBase.emitSuccess(FutureBase.java:60) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.SucceededFuture.addListener(SucceededFuture.java:88) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.Composition.onSuccess(Composition.java:43) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.FutureBase.emitSuccess(FutureBase.java:60) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.FutureImpl.tryComplete(FutureImpl.java:211) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.PromiseImpl.tryComplete(PromiseImpl.java:23) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.PromiseImpl.onSuccess(PromiseImpl.java:49) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.FutureBase.lambda$emitSuccess$0(FutureBase.java:54) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) [io.netty.netty-common-4.1.68.Final.jar:4.1.68.Final] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469) [io.netty.netty-common-4.1.68.Final.jar:4.1.68.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) [io.netty.netty-transport-4.1.68.Final.jar:4.1.68.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) [io.netty.netty-common-4.1.68.Final.jar:4.1.68.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [io.netty.netty-common-4.1.68.Final.jar:4.1.68.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty.netty-common-4.1.68.Final.jar:4.1.68.Final] at java.lang.Thread.run(Thread.java:829) [?:?] 2022-05-23 05:15:04 WARN AbstractOperator:481 - Reconciliation #335(timer) KafkaMirrorMaker2(kafka-cold/strimzi-kafka-to-cold-mirror): Failed to reconcile java.lang.NullPointerException: null at io.strimzi.operator.common.Util.lambda$authTlsHash$10(Util.java:558) ~[io.strimzi.operator-common-0.26.0.jar:0.26.0] at io.vertx.core.impl.future.Composition.onSuccess(Composition.java:38) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.FutureBase.emitSuccess(FutureBase.java:60) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.FutureImpl.tryComplete(FutureImpl.java:211) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.Composition$1.onSuccess(Composition.java:62) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.FutureBase.emitSuccess(FutureBase.java:60) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.SucceededFuture.addListener(SucceededFuture.java:88) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.Composition.onSuccess(Composition.java:43) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.FutureBase.emitSuccess(FutureBase.java:60) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.FutureImpl.tryComplete(FutureImpl.java:211) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.PromiseImpl.tryComplete(PromiseImpl.java:23) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.PromiseImpl.onSuccess(PromiseImpl.java:49) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.vertx.core.impl.future.FutureBase.lambda$emitSuccess$0(FutureBase.java:54) ~[io.vertx.vertx-core-4.1.5.jar:4.1.5] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) [io.netty.netty-common-4.1.68.Final.jar:4.1.68.Final] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469) [io.netty.netty-common-4.1.68.Final.jar:4.1.68.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500) [io.netty.netty-transport-4.1.68.Final.jar:4.1.68.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) [io.netty.netty-common-4.1.68.Final.jar:4.1.68.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [io.netty.netty-common-4.1.68.Final.jar:4.1.68.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty.netty-common-4.1.68.Final.jar:4.1.68.Final] at java.lang.Thread.run(Thread.java:829) [?:?]

Additional context not much

[scholzj]

BTW: Checking the code, it looks like 0.29 already returns a much nicer error instead of the NPE whcih is not tht useful without digging through the code.

[sunshine69]

Right I found what changes I added to the secret has been wiped off, so it might be it. Closing this ticket for now. Thanks for your help.