Prometheus fails to scrape Helidon metrics

Environment Details

• Helidon Version: 0.10.1
• Helidon SE
• JDK version: 1.8.0_162
• OS: MacOS

---

Problem Description

Prometheus 2.2.1 and 2.4.3 fail to scrape Helidon metrics. Prometheus reports the error no token found. This error is typically reported by Prometheus when it can’t parse the scrapped metrics data.

Steps to reproduce

1. Build and run this Helidon app: https://github.com/barchetta/helidon-se-codeone-2018
2. Download and run Prometheus 2.2.1 (https://github.com/prometheus/prometheus/releases/tag/v2.2.1)
3. Add a scrape_configs to prometheus.yml like this:

  - job_name: 'helidon'

    metrics_path: '/metrics/'
    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ['localhost:8080']

You’ll see prometheus report:

level=warn ts=2018-10-16T01:35:44.714172501Z caller=scrape.go:697 component="scrape manager" scrape_pool=helidon target=http://localhost:8080/metrics/ msg="append failed" err="no token found"

[barchetta]

We have confirmation from MicroProfile that the spec is misleading: https://groups.google.com/forum/?utm_source=digest&utm_medium=email#!topic/microprofile/t4YgzPpmOsA

"I’m part of the metrics community and I agree that what’s specified in section 2.3, related to the Accept header isn’t as specific as it ought to be. The way it’s worded in the spec right now isn’t right, as illustrated by Joe’s example.

The way we’d intended for it to be interpreted was that if the incoming request accepts text/plain with a higher priority than application/json, then you should use Prometheus format. If the incoming request accepts application/json with a higher priority than text/plain, then use JSON format. If the incoming request doesn’t accept either of those, then return a 406 status code."

There is now an issue to clarify this in the spec: eclipse/microprofile-metrics#291

[spericas]

Interesting, I agree. Take a look at HashRequestHeaders::bestAccepted.