question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Lots of duplicate metric definitions for micrometer shim

See original GitHub issue

If you run a simple spring boot app with spring-boot-starter-actuator and with the micrometer1-shim, you get lots of duplicate metric definition warnings like:

14:57:23.351 [main] WARN  io.opentelemetry.sdk.metrics.internal.state.MetricStorageRegistry - Found duplicate metric definition: jvm.threads.states
        at unknown source
                To enable better debugging, run your JVM with -Dotel.experimental.sdk.metrics.debug=true
Causes
- Description [The current number of threads having RUNNABLE state] does not match [The current number of threads having NEW state]
- InstrumentDescription [The current number of threads having RUNNABLE state] does not match [The current number of threads having NEW state]
Original instrument registered with same name but is incompatible.
        at unknown source
                To enable better debugging, run your JVM with -Dotel.experimental.sdk.metrics.debug=true

14:57:23.352 [main] WARN  io.opentelemetry.sdk.metrics.internal.state.MetricStorageRegistry - Found duplicate metric definition: jvm.threads.states
        at unknown source
                To enable better debugging, run your JVM with -Dotel.experimental.sdk.metrics.debug=true
....

I enabled better debugging with Dotel.experimental.sdk.metrics.debug=true and found that the issue seems to be an incompatibility between the micrometer definition of a metric description and the opentelemetry description. In micrometer it appears that each unique set of tags on an instrument can have its own description. Check out the JvmThreadMetrics for an example.

In opentelemetry an instrument is defined by its name, type, description, and unit. If an instrument with the same name, but a different type, description, or unit than one previously created, you’ve created a semantic error an we log the warnings I’ve included on this ticket.

We should probably drop micrometer instrument descriptions to ensure we don’t create these semantic errors. The unit can be different for each set of tags as well, so we should consider dropping that as well. However, in practice I didn’t see any conflicts created by differing units.

Issue Analytics

  • State:closed
  • Created a year ago
  • Reactions:2
  • Comments:5 (4 by maintainers)

github_iconTop GitHub Comments

1reaction
jonatan-ivanovcommented, May 2, 2022

@jack-berg Based on user feedback, we decided to postpone 2.x indefinitely. This can be “weird” for other registries as well, e.g.: Prometheus, where the description is for a metric group so if there are multiple descriptions (one for each time series), you will only see one (the first one I guess).

We can try to improve this by removing these dynamic parts from the micrometer instrumentation (i.e.: the thread state metrics you mentioned) but this won’t fix other instrumentation or when the user sets the description dynamically (though these cases should be rare).

@rafaribe @jack-berg A short term solution can be adding a MeterFilter that can change the description for the problematic Meters.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Metric_instance records are being duplicated - ServiceNow
But I am seeing duplicate metric records (different sys_id but same field values) . ... var mi = new MetricInstance(definition, current);.
Read more >
Recommended best practices for collecting, analyzing, and ...
Lessons from a comprehensive regional study of microplastics in San Francisco Bay. •. We offer recommendations on collection, analysis, ...
Read more >
Blog | Prometheus
As such, test shims are defined as being passed. ... building a custom implementation to collect our metrics using the Micrometer interface.
Read more >
ELEMENTS OF METRIC GEAR TECHNOLOGY
that this be a source for all information about gearing with definition in a metric format. 1.1 Comparison Of Metric Gears With American...
Read more >
Length and Dimensional Measurements at NIST - PMC - NCBI
Three different definitions of the international standard of length have been ... up to 10 metric tons needed to be made at time...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found