question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Exporter not picking up AWS/Kafka metrics

See original GitHub issue

I am able to get metrics using the exporter tool for other AWS resources but when attempting to use for Kafka, no metrics are being retrieved:

`HELP tagging_api_requests_total API requests made to the Resource Groups Tagging API TYPE tagging_api_requests_total counter

HELP cloudwatch_exporter_scrape_duration_seconds Time this CloudWatch scrape took, in seconds.

TYPE cloudwatch_exporter_scrape_duration_seconds gauge cloudwatch_exporter_scrape_duration_seconds 0.0403737

HELP cloudwatch_exporter_scrape_error Non-zero if this scrape failed.

TYPE cloudwatch_exporter_scrape_error gauge

cloudwatch_exporter_scrape_error 0.0

HELP cloudwatch_requests_total API requests made to CloudWatch

TYPE cloudwatch_requests_total counter

cloudwatch_requests_total{action=“listMetrics”,namespace=“AWS/Kafka”,} 16.0`

There is not any examples of using Kafka in the repository, does anyone know if it has been successfully used for Kafka before?

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
vasremcommented, Feb 4, 2020

BTW guys, JFYI you have to specify both aws_dimensions to get the metrics back as @RmMsr described.

1reaction
RmMsrcommented, Jan 28, 2020

Hi @jim5252 ,

we experienced the same problem. After quite a while of experimenting we noticed that the naming is crucial. For example the dimensions must be identical to the AWS naming with case and whitespace.

Example:

metrics:
  - aws_namespace: AWS/Kafka
    aws_metric_name: KafkaDataLogsDiskUsed
    aws_dimensions: [Broker ID, Cluster Name]
    aws_statistics: [Maximum]
Read more comments on GitHub >

github_iconTop Results From Across the Web

Open monitoring with Prometheus - AWS Documentation
You can monitor your MSK cluster with Prometheus, an open-source monitoring system for time-series metric data. You can publish this data to Amazon...
Read more >
Prometheus Cloudwatch-Exporter not able to fetch metrics ...
New to Prometheus, trying to configure cloudwatch-exporter to pull metrics from AWS account. Exporter is running elb.
Read more >
Connecting Grafana to Prometheus - MSK Labs
The occasional error isn't an issue, it's when you see a spike/growth in errors while the requests/sec drop that you need to investigate....
Read more >
Kafka | Sysdig Documentation
Metrics, Dashboards, Alerts and more for Kafka Integration in Sysdig Monitor. ... [Kafka] Broker Down, There are less Kafka brokers up than expected....
Read more >
Amazon MSK (Agent) - Datadog Docs
Amazon Managed Streaming for Apache Kafka (MSK) is a fully managed service that makes it ... directory to start collecting your Amazon MSK...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found