JMX_Exporter not responding

I seem to be getting a similar issue as: https://github.com/prometheus/jmx_exporter/issues/487

jmx_exporter version: 0.16.1

ELK config: -javaagent:/opt/jmxexporter/jmx_prometheus_javaagent-0.16.1.jar=7071:/opt/jmxexporter/elasticsearch-jmx.yml

elasticsearch-jmx.xml:

lowercaseOutputName: true

Once I restart ELK and try to curl -v localhost:7071/ it works the first 1-2 times (I get legitimate metrics) but then it stops and I only get “Connection reset” errors which indicates the jar crashed (?).

Any way to debug this further?

(please don’t repoint me to the prometheus mailing list, I believe this looks like an exporter issue and not related to prometheus directly)

Many thanks in advance! 👍

[fstab]

Thanks for the detailed report. As you have whitelisted java.lang:type=OperatingSystem it would be good to know:

• What operating system are you using? Maybe there’s something unexpected in your OS metrics. Let me know what your system is, then I’ll try to reproduce it.
• Does this have anything to do with Elastic, or does this happen with any Java application on that system? The integration tests in jmx_exporter come with a simple example application. If you build jmx_exporter from source with ./mvnw package the example application will be created in ./integration_tests/jmx_example_application/target/jmx_example_application.jar. It would be great if you could run the example application instead of Elastic with the Java agent attached to see if you get the same behavior:

  java -javaagent:/opt/jmx_exporter/jmx_prometheus_javaagent-0.17.0.jar=8088:/opt/jmx_exporter/java_process.yml -jar jmx_example_application.jar
  

[fstab]

Here’s an idea how to debug this: You could try to figure out whether this is caused by a specific JMX bean.

Use the whitelistObjectNames configuration so that jmx_exporter just scrapes a single bean. If that works, add more names to whitelistObjectNames until you learn which bean causes this.