Prometheus plugins exception in log results in empty page when scraping
See original GitHub issueHello,
Steps to reproduce
Use prometheus-plugin: 2.0.8 and Jenkins: 2.235.3 with a large number of jobs and scrape them, an empty scraping page will be returned
Expected behaviour Show the scraped output on the /prometheus endpoint
Actual behaviour When navigating to the webpage/prometheus, I am receiving an empty page with a 200 status code. In my logs I see:
A thread (prometheus_async_worker thread/767) died unexpectedly due to an uncaught exception, this may leave your Jenkins in a bad way and is usually indicative of a bug in the code.
java.lang.NullPointerException
at org.jenkinsci.plugins.prometheus.JobCollector.lambda$appendJobMetrics$1(JobCollector.java:310)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1699)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
at org.jenkinsci.plugins.prometheus.JobCollector.appendJobMetrics(JobCollector.java:310)
at org.jenkinsci.plugins.prometheus.JobCollector.lambda$collect$0(JobCollector.java:220)
at org.jenkinsci.plugins.prometheus.util.Jobs.forEachJob(Jobs.java:14)
at org.jenkinsci.plugins.prometheus.JobCollector.collect(JobCollector.java:214)
at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.findNextElement(CollectorRegistry.java:190)
at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:223)
at io.prometheus.client.CollectorRegistry$MetricFamilySamplesEnumeration.nextElement(CollectorRegistry.java:144)
at io.prometheus.client.exporter.common.TextFormat.write004(TextFormat.java:22)
at org.jenkinsci.plugins.prometheus.service.DefaultPrometheusMetrics.collectMetrics(DefaultPrometheusMetrics.java:51)
at org.jenkinsci.plugins.prometheus.service.PrometheusAsyncWorker.execute(PrometheusAsyncWorker.java:41)
at hudson.model.AsyncPeriodicWork.lambda$doRun$0(AsyncPeriodicWork.java:100)
at java.lang.Thread.run(Thread.java:748)
I am also seeing
A thread (prometheus_async_worker thread/1054) died unexpectedly due to an uncaught exception, this may leave your Jenkins in a bad way and is usually indicative of a bug in the code.
java.lang.NullPointerException
In the prometheus logs, it seems to still receive information correctly from jobs, but I got no output when jenkins is scraped.
Server configuration prometheus-plugin: 2.0.8 Jenkins: 2.235.3
Settings for the plugins is the default settings.
Notify @markyjackson-taulia
Issue Analytics
- State:
- Created 3 years ago
- Reactions:2
- Comments:6
Top Results From Across the Web
Prometheus API return blank page · Issue #82 - GitHub
As I try to setup the log with org.jenkinsci.plugins.prometheus. Any thing prints out, so I guess the prometheus plugin doesn't execute.
Read more >Writing exporters - Prometheus.io
The one exception is when you're exporting the same data with different labels via multiple metrics, in which case that's usually the sanest...
Read more >4. Exposition - Prometheus: Up & Running [Book] - O'Reilly
Each time Prometheus went to scrape the application, it would randomly get the metrics from only one of the workers, which would be...
Read more >Monitor your logging pipeline with Prometheus Operator
Monitor your logging pipeline with Prometheus Operator. You can configure the Logging operator to expose metrics endpoints for FluentD and Fluent-bit using ...
Read more >How to export metrics (collected by DropWizard) in ...
I have implemented different ways to export metrics to Prometheus: 1.1) Custom implementation for Pushgateway.
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
Similar issue here. Since plugin 2.0.8 upgrade: There is a ClassCastException in the log and the /prometheus page is empty
SEVERE h.i.i.InstallUncaughtExceptionHandler$DefaultUncaughtExceptionHandler#uncaughtException: A thread (prometheus_async_worker thread/3301139) died unexpectedly due to an uncaught exception, this may leave your Jenkins in a bad way and is usually indicative of a bug in the code. java.lang.ClassCastException
Note: As soon as I disable the five “Count Duration *” settings of the plugin in “Manage Jenkins”, the error log entry is gone and the (now less) metrics are back.
Stale issue message