question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

JVM crash with Java 7 + Agent 1.3.0

See original GitHub issue

Describe the bug We updated apm agent to 1.3.0 yesterday and then after about 18 hours two instances got JVM crash which look pretty similar.

# A fatal error has been detected by the Java Runtime Environment:
#  SIGSEGV (0xb) at pc=0x000000008fb018d8, pid=122146, tid=140043922573056

Both are using Java 7 (7.0_75 and 7.0_141). One is running on Glassfish 4.0 and the other Payara 4.1.1.171.1.

Debug logs

Current thread (0x00007f7ee0354800):  JavaThread "apm-reporter" daemon 

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  0x000000008fb018d8

Also there is co.elastic.apm.agent.report.serialize.DslJsonSerializer::sanitizeTagKey in both compilation events.

Full logs: hs_err_pid122146.log hs_err_pid43918.log

This might be related to elastic/apm-agent-java#444

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:47 (23 by maintainers)

github_iconTop GitHub Comments

1reaction
Aketzucommented, Feb 12, 2019

About 18h uptime now with 1.3.1-SNAPSHOT without any compile exclusions so seems to work.

1.3.0 with exclusions still ended up crashing after 17h uptime. Last compiled method was java.lang.invoke.BoundMethodHandle$Species_LL::<init> and nothing else obvious to my eye. Really strange how well it breaks JVM 😃 hs_err_pid27244.log test-20190210_184348.log

1reaction
felixbarnycommented, Feb 11, 2019

I think we should use reflection instead of method handles if the agent is running on Java 7.

Read more comments on GitHub >

github_iconTop Results From Across the Web

JDK-8206397 JVM crash in C [libzip.so+0x12b33 ... - Bug ID
JDK -8206397 : JVM crash in C [libzip.so+0x12b33] ZIP_GetNextEntry+0x53 while running Weblogic 12.1.3. Type: Bug; Component: core-libs; Sub-Component: ...
Read more >
APM agent for Java is causing system crashes - Elastic Discuss
The APM-Java is working but at some point Tomcat crashes. A fatal error has been detected by the Java Runtime Environment: Reason to...
Read more >
Java™ SE Development Kit 7 Update 40 Bug Fixes - Oracle
Bug ID Component Description 8001161 client‑libs EmbeddedFrame doesn't become active window 8004316 client‑libs Printing an image using AUTOSENSE fails to print 8015375 client‑libs Edits to text...
Read more >
Problem determination for javacore files from WebSphere ...
Some error conditions in an IBM® Java™ virtual machine (JVM) running under IBM WebSphere® Application Server result in a crash.
Read more >
Compatibility and requirements for the Java agent
JVM ; Java 8. v3.10.0 to current. v4.9.0 and lower ; Java 9. v3.43.0 to current. v4.9.0a and lower ; Java 10. v4.4.0...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found