Kafka Exporter has **CrashLoopBackOff** and can't recover.

Please use this to only for bug reports. For questions or when you need help, you can use the GitHub Discussions, our #strimzi Slack channel or out user mailing list.

Describe the bug Kafka Exporter has CrashLoopBackOff and can’t recover.

To Reproduce Sometimes, when I create Kafka Cluster and it works. But after a while, the Kafka Exporter has CrashLoopBackOff and is always in this status.

Readiness probe failed: Get "http://10.130.0.49:9404/metrics": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Expected behavior A clear and concise description of what you expected to happen.

Environment (please complete the following information):

• Strimzi version: main
• Installation method: OperatorHub
• Kubernetes cluster: Openshift 4.9
• Infrastructure: BareMetal

YAML files and logs

[kafka_exporter] [INFO] 2022/01/04 07:38:01 Starting kafka_exporter (version=1.3.1.redhat-00001, branch=master, revision=eb1f5c4229ce4ca51d64d2034926ce64c60e05e9)
[kafka_exporter] [INFO] 2022/01/04 07:38:01 Build context (go=go1.13, user=worker@pnc-ba-pod-4c2d6e, date=20210708-16:03:34)
[kafka_exporter] [INFO] 2022/01/04 07:38:01 Done Init Clients
[kafka_exporter] [INFO] 2022/01/04 07:38:01 Listening on :9404
[kafka_exporter] [INFO] 2022/01/04 07:38:02 Refreshing client metadata
[kafka_exporter] [INFO] 2022/01/04 07:38:05 concurrent calls detected, waiting for first to finish
[kafka_exporter] [INFO] 2022/01/04 07:38:17 concurrent calls detected, waiting for first to finish
[kafka_exporter] [INFO] 2022/01/04 07:38:32 concurrent calls detected, waiting for first to finish
[kafka_exporter] [INFO] 2022/01/04 07:38:35 concurrent calls detected, waiting for first to finish
[kafka_exporter] [INFO] 2022/01/04 07:38:45 concurrent calls detected, waiting for first to finish
[kafka_exporter] [INFO] 2022/01/04 07:38:47 concurrent calls detected, waiting for first to finish
[kafka_exporter] [INFO] 2022/01/04 07:39:05 concurrent calls detected, waiting for first to finish
[kafka_exporter] [INFO] 2022/01/04 07:39:15 concurrent calls detected, waiting for first to finish
[kafka_exporter] [INFO] 2022/01/04 07:39:15 concurrent calls detected, waiting for first to finish

[scholzj]

Triaged on 2.8.2022: This has to be fixed in the Kafka Exporter. Once it has a new release, we can update Strimzi to use it. We should also add a warning to the docs about this problem (=> e.g. something like If you don’t use consumer groups, it will not work … just with more fancy wording 😮). It can be added for example somewhere here: https://strimzi.io/docs/operators/latest/deploying.html#con-metrics-kafka-exporter-lag-str

CC @PaulRMellor ^^^

We need to also re-open the discussion about the Kafka Exporter future since it is a long time since a new release or some fixed issues.

[scholzj]

I don’t think it really uses the consumer groups. It just reads and decodes the content of it. So it does not trigger its creation.