question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Optionally serve metrics in protocol buffer format?

See original GitHub issue

Thank for the great library!

Would you be interested in a pull request related to serving metrics to prometheus in the protocol buffer format?

My thought was to create a new function in lib/register.js named something like getMetricsAsProtobuf() which returns a buffer containing the 32-bit varint-encoded record length-delimited protocol buffer messages of type io.prometheus.client.MetricFamily which prometheus expects. This buffer could be returned in the body of an HTTP response using something like Express or Koa.

There would be one added dependency, protobufjs.

The only issue is, you guys internally store the metrics in a format that is not compliant with prometheus’ metrics.proto messages, so either A), a conversion function is required, or B), we consider updating prom-client to internally store metrics as MetricFamily compliant objects. (E.G. for metric.type, we store 0 instead of counter, etc.)

Here’s the conversion function we’re using, it would be nice not to have to do this. It returns an array of metric objects from which we create protobuf messages using protobufjs’s generated code’s MetricFamily.fromObject(metric) function. (Note: support for Summary and Histogram is unfinished.)

// Convert prom-client metric objects to `io.prometheus.client.MetricFamily` compliant objects
function convertMetrics() {
    let convertedMetrics = []
    for (let metricFamily of prom.getMetricsAsJSON()) {
        let newMetricFamily = {}
        newMetricFamily.name = metricFamily.name
        newMetricFamily.help = metricFamily.help
        let newMetrics = []
        for (let metric of metricFamily.values) {
            let newMetric = {}
            let newLabels = []
            Object.keys(metric.labels).forEach(function (key) {
                if (metric.labels[key].toString() != null)
                    newLabels.push({ "name": key, "value": metric.labels[key].toString() });
            });
            newMetric.label = newLabels;
            switch (metricFamily.type) {
                case "counter":
                    newMetricFamily.type = 0;
                    newMetric.counter = { "value": metric.value };
                    break;
                case "gauge":
                    newMetricFamily.type = 1;
                    newMetric.gauge = { "value": metric.value };
                    break;
                case "summary":
                    newMetricFamily.type = 2;
                    // TODO
                    break;
                case "histogram":
                    newMetricFamily.type = 4;
                    // TODO
                    break;
            }
            // TODO timestamp_ms is not supported in prom-client? (confirm or contribute)
            // newMetric.timestamp_ms = metric.timestamp_ms;
            newMetrics.push(newMetric);
        }
        newMetricFamily.metric = newMetrics
        convertedMetrics.push(newMetricFamily);
    }
    return convertedMetrics;
}

I’ve written code that successfully sends the protobuf messages to prometheus, I’d like to share it, but the conversion part feels hacky. Thoughts?

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Reactions:1
  • Comments:10 (3 by maintainers)

github_iconTop GitHub Comments

2reactions
brian-brazilcommented, Mar 26, 2018

Prometheus 2.x does not support proto, so this can be closed.

1reaction
SimenBcommented, Apr 19, 2017

This would be really nice! Timestamp is #69 btw (the todo in your sample code)

Read more comments on GitHub >

github_iconTop Results From Across the Web

Language Guide | Protocol Buffers - Google Developers
First let's look at a very simple example. Let's say you want to define a search request message format, where each search request...
Read more >
Prometheus protobuf message question (issue# 547)
- A MetricFamily are all Metrics of the same name. In my understanding, as per text exposition format, only 1 HELP and optionally...
Read more >
Protobuf definition best practices | by Ammar Khaku - Medium
Protobuf messages define structured objects. As a service owner, you can be sure that data you are receiving comes parsed into a specific ......
Read more >
OpenTelemetry Protocol Specification
This specification defines how OTLP is implemented over gRPC and HTTP 1.1 transports and specifies Protocol Buffers schema that is used for the...
Read more >
5 Reasons to Use Protocol Buffers Instead of JSON for Your ...
“Protocol Buffers are a way of encoding structured data in an efficient yet extensible format.” Google developed Protocol Buffers for use in ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found