question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

DatadogMeterRegistry StackOverflowError

See original GitHub issue

Hi,

we receive endless Stackoverflow exceptions if the datadog endpoint is not available anymore for any reason or they have to restart the servers or something:

The message in DD lookes like this:

{"timestamp":"2019-03-18T23:47:21.187+00:00","message":"failed to send metrics to datadog","logger":"i.m.datadog.DatadogMeterRegistry","thread_name":"datadog-metrics-publisher","status":"WARN","stack_trace":"java.lang.StackOverflowError: null\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1044)\n\tat java.base/java.util.Collections$UnmodifiableCollection$1.hasN

I don’t see any logs other than this, no warning that metrics couldn’t be sent or any other exception.

If i look into DatadogMeterRegistry i just see only one line where “hasNext()” is used, maybe you can find the loop in your code.

We’r using “micrometer-registry-datadog-1.1.2.jar” which is shipped with spring-boot 2.1

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:15 (6 by maintainers)

github_iconTop GitHub Comments

1reaction
jens-meisscommented, Apr 3, 2019

ok, we’ll wait for the next jdk-11 updates and see if we’ll have the problem again, we had the problem only 2 times in the past 4 months, maybe it will take a while until we see it again …

@olamelin thanks for you investigation @izeye you can close the ticket if you want to, i’ll open a new one if we see the error again after the jdk-11.0.3 release

1reaction
olamelincommented, Apr 1, 2019

Since my last comment I created a parallel image of our code that was based on a openjdk12 image. In order to see if the JDK issue was the cause. The one using openjdk12 has not reported the stack overflow the last 6 days but the parallel image (which was restarted) has shown this error multiple times. In other words I think it is safe to say that this issue is caused by the bug described here https://bugs.openjdk.java.net/browse/JDK-8214129

Read more comments on GitHub >

github_iconTop Results From Across the Web

java - Custom metrics sent datadog using micrometer ...
I want to send custom metrics using io.micrometer.datadog.DatadogMeterRegistry to datadog. Below is the code snippet ...
Read more >
Custom metrics sent datadog using micrometer ...
Coding example for the question Custom metrics sent datadog using micrometer DatadogRegistry not showing up in Datadog metric summary-Springboot.
Read more >
Hi I have added metrics to an application like this install | Ktor ...
Hi I have added metrics to an application like this install MicrometerMetrics registry DatadogMeterRegistry config Clock SYSTEM Would it be possible to ...
Read more >
DatadogMeterRegistry (micrometer-registry-datadog 1.1.5 API)
DatadogMeterRegistry. @Deprecated public DatadogMeterRegistry(DatadogConfig config, io.micrometer.core.instrument.Clock clock, java.util.concurrent.
Read more >
v1.0.7 · 标签 · mirrors / micrometer-metrics / micrometer · GitCode
... (fixes #789) - 9471e3f1fe0409de9f3b95ac022232efff8719eb: Fix StackOverflow in ... Log an error with invalid URI string in DatadogMeterRegistry.publish() ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found