Unable to fetch the CPU usage metrics in elastic search for Openshift JVMs
See original GitHub issueDear Team,
we are unable to get the CPU metrics for our applications, we could see rest of all the other metrics, but not the CPU, this is a crucial parameter to assess the impact to application
Below are the details.
Elastic Search version: 7.17.1 Elastic APM Jar version: 1.30.0 Spring boot version: spring-boot-2.3.10.RELEASE OpenShift Master: v3.11.0+ec8630f-265 Kubernetes Master: v1.11.0+d4cacc0 OpenShift Web Console: v3.11.0+ea42280 Java: 11.0.14.1 (eclipse temurin)
ERROR/WARN:
2022-04-05 18:46:08,628 [main] WARN co.elastic.apm.agent.impl.circuitbreaker.SystemCpuStressMonitor - Failed to obtain reference to the getSystemCpuLoad method of this JVM's OperatingSystemMXBean implementation
Originating stack trace
Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport
2022-04-05 18:46:04,825 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - Starting Elastic APM 1.30.0 as link-api-gateway on Java 11.0.14.1 Runtime version: 11.0.14.1+1 VM version: 11.0.14.1+1 (Eclipse Adoptium) Linux 3.10.0-1062.18.1.el7.x86_64
2022-04-05 18:46:04,829 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - service_name: 'my-api-gateway' (source: Environment Variables)
2022-04-05 18:46:04,830 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - environment: 'DEV' (source: Environment Variables)
2022-04-05 18:46:04,830 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - enable_experimental_instrumentations: 'true' (source: Java System Properties)
2022-04-05 18:46:04,830 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - server_url: 'https://eapm-nonprod-dmz.xxxxxxx.com:443' (source: Environment Variables)
2022-04-05 18:46:04,830 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - application_packages: 'com.xxxxxx.my.api.gateway' (source: Environment Variables)
2022-04-05 18:46:08,628 [main] WARN co.elastic.apm.agent.impl.circuitbreaker.SystemCpuStressMonitor - Failed to obtain reference to the getSystemCpuLoad method of this JVM's OperatingSystemMXBean implementation
2022-04-05 18:46:08,634 [main] INFO co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to RUNNING state
2022-04-05 18:46:10,113 [elastic-apm-server-healthcheck] INFO co.elastic.apm.agent.report.ApmServerHealthChecker - Elastic APM server is available: { "build_date": "2022-02-23T22:31:58Z", "build_sha": "990c162bf1d5106be95d1430c6f3e98df865df7d", "publish_ready": true, "version": "7.17.1"}
2022-04-05 18:46:10,211 [elastic-apm-remote-config-poller] INFO co.elastic.apm.agent.configuration.ApmServerConfigurationSource - Received new configuration from APM Server: {}
#logback.classic pattern: %d %-5level [%thread] %logger{0}: %msg%n
#logback.classic pattern: %d %-5level [%thread] %logger{0}: %msg%n
#logback.classic pattern: %d %-5level [%thread] %logger{0}: %msg%n
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.3.10.RELEASE)
Please let me know for more informatin.
Issue Analytics
- State:
- Created a year ago
- Comments:6 (2 by maintainers)
Top Results From Across the Web
Troubleshooting for Critical Alerts | Logging
Check if the Elasticsearch JVM Heap usage is high. oc exec -n openshift-logging -c elasticsearch <elasticsearch_pod_name> -- es_util --query=_nodes/stats?pretty.
Read more >Elasticsearch High CPU Usage - Main Causes and Solutions
High CPU is generally the consequence of JVM garbage collection, which in turn is caused by configuration or query related issues. In a...
Read more >Chapter 12. Troubleshooting Logging OpenShift Container ...
Check if the Elasticsearch JVM Heap usage is high.
Read more >How to Collect Elasticsearch Metrics - Datadog
There are five main HTTP APIs that you can use to monitor Elasticsearch: Node Stats API; Cluster Stats API; Index Stats API; Cluster...
Read more >Manage compute resources | Elastic Cloud on Kubernetes [2.5]
Starting with Elasticsearch 7.11, the heap size of the JVM is automatically ... The two relevant metrics are kubernetes.pod.cpu.usage.limit.pct for CPU, ...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
@jackshirazi , Marvellous!, This fixed the issue, Thank you so much!!!.
@eyalkoren and @hakansel , Issue is resolved now by adding jdk.management to our JRE
Issue can be closed. Thank you.
Can you try adding
jdk.management
?