Cruise Control getting stuck in Bootstrapping
See original GitHub issueHello,
I am trying to install Cruise-Control for monitoring our Kafka Environment.
I have a distributed Kafka Environment with 3 VMs (Linux OS) and each VM has one Kafka Broker and one Zookeeper in it.
So in the cruisecontrol.properties
file I have:
bootstrap.servers=my-domain-dev01.com:9093,my-domain-dev02.com:9094,my-domain-dev03.com:9095
where each my-domain-dev#
is a separate VM.
I also have the following for the Zookeeper:
zookeeper.connect=my-domain-dev01.com:2183,my-domain-dev02.com:2184,my-domain-dev03.com:2185
At this point I am able to see Kafka Cluster State
just fine but when it comes to Metrics
I am having the following issue:
[2020-10-08 10:07:53,467] INFO Finished sampling in 254 ms. (com.linkedin.kafka.cruisecontrol.monitor.sampling.MetricFetcherManager)
[2020-10-08 10:07:53,468] INFO Kicking off metric sampling for time range [770800000, 771000000], duration 200000 ms with timeout 200000 ms. (com.linkedin.kafka.cruisecontrol.monitor.sampling.MetricFetcherManager)
[2020-10-08 10:07:53,495] INFO [Consumer clientId=CruiseControlMetricsReporterSampler-8036022001223885340-consumer-833857043, groupId=CruiseControlMetricsReporterSampler-8036022001223885340] Seeking to offset 22102016 for partition __CruiseControlMetrics-0 (org.apache.kafka.clients.consumer.KafkaConsumer)
[2020-10-08 10:07:53,702] INFO Finished sampling for topic partitions [__CruiseControlMetrics-0] in time range [770800000,771000000]. Collected 0 metrics. (com.linkedin.kafka.cruisecontrol.monitor.sampling.CruiseControlMetricsReporterSampler)
[2020-10-08 10:07:53,702] INFO Collected 0 partition metric samples for 0 partitions. Total partition assigned: 218. (com.linkedin.kafka.cruisecontrol.monitor.sampling.SamplingFetcher)
[2020-10-08 10:07:53,702] INFO Collected 0 broker metric samples for 0 brokers. (com.linkedin.kafka.cruisecontrol.monitor.sampling.SamplingFetcher)
As you can see it is not collecting any metrics and I noticed that the timestamps are way off:
time range [770800000, 771000000]
= time range [June 5, 1994 7:06:40 AM, June 7, 1994 2:40:00 PM]
Could this be the issue? Is there a way to fix this? What else could I look into?
I can see records being populated in __CruiseControlMetrics
continuously but not __KafkaCruiseControlPartitionMetricSamples
or __KafkaCruiseControlPartitionMetricSamples
.
I would appreciate any guidance on this.
Thank you.
Issue Analytics
- State:
- Created 3 years ago
- Comments:7 (3 by maintainers)
Top GitHub Comments
@efeg ok, I was using the CC UI Bootstrap “Metric button”. I wasn’t aware that this is the normal behavior.
Thanks
@jrevillard What does it mean to click on
Bootstrap
? Are you using the bootstrap endpoint of Cruise Control (CC)? If so, this endpoint is used only for development purposes and is not really meant to be used for bootstrapping a CC instance. When CC starts, it automatically bootstraps w/o the need for any extra call.