Classgraph causes OpenJDK 11 VM to crash

Hi there,

we are running into an issue where classpath scanning on Java 11 cause the JVM to crash.

Not sure exactly what is causing the crash as we run this test through Junit5 and Jacoco agent but it appears to be some issue with Classgraph looking through jigsaw modules.

<dependency>
  <groupId>io.github.classgraph</groupId>
  <artifactId>classgraph</artifactId>
  <version>4.8.27</version>
</dependency>

OpenJDK we use:

$ java --version
openjdk 11.0.3 2019-04-16 LTS
OpenJDK Runtime Environment Corretto-11.0.3.7.1 (build 11.0.3+7-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.3.7.1 (build 11.0.3+7-LTS, mixed mode)

Full file here:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x000000010305ca9b, pid=83655, tid=8451
#
# JRE version: OpenJDK Runtime Environment (11.0.3+7) (build 11.0.3+7-LTS)
# Java VM: OpenJDK 64-Bit Server VM (11.0.3+7-LTS, mixed mode, tiered, compressed oops, g1 gc, bsd-amd64)
# Problematic frame:
# V  [libjvm.dylib+0x65ca9b]  ResolvedMethodTable::add_method(Handle)+0x61
#
# Core dump will be written. Default location: /cores/core.83655
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#

---------------  S U M M A R Y ------------

Command Line: --illegal-access=warn -javaagent:/Users/fred/.m2/repository/org/jacoco/org.jacoco.agent/0.8.3/org.jacoco.agent-0.8.3-runtime.jar=destfile=/Users/fred/workspaces/wiremock-extension/target/jacoco.exec,append=true /Users/fred/workspaces/wiremock-extension/core/target/surefire/surefirebooter4586396533766981393.jar /Users/fred/workspaces/wiremock-extension/core/target/surefire 2019-04-30T14-10-32_340-jvmRun1 surefire3472493790080823102tmp surefire_05804185634312275564tmp

Host: MacBookPro14,3 x86_64 2800 MHz, 8 cores, 16G, Darwin 18.0.0
Time: Tue Apr 30 14:10:40 2019 AEST elapsed time: 7 seconds (0d 0h 0m 7s)

---------------  T H R E A D  ---------------

Current thread (0x00007f8b35001000):  JavaThread "main" [_thread_in_vm, id=8451, stack(0x000070000df25000,0x000070000e025000)]

Stack: [0x000070000df25000,0x000070000e025000],  sp=0x000070000e01dfc0,  free space=995k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.dylib+0x65ca9b]  ResolvedMethodTable::add_method(Handle)+0x61
V  [libjvm.dylib+0x4eabf5]  CallInfo::set_resolved_method_name(Thread*)+0x5d
V  [libjvm.dylib+0x4eaaf7]  CallInfo::CallInfo(Method*, Klass*, Thread*)+0x1c5
V  [libjvm.dylib+0x3a95e5]  java_lang_StackFrameInfo::set_method_and_bci(Handle, methodHandle const&, int, Thread*)+0x7f
V  [libjvm.dylib+0x68f6f0]  StackWalk::fill_in_frames(long, BaseFrameStream&, int, int, objArrayHandle, int&, Thread*)+0x252
V  [libjvm.dylib+0x6907d3]  StackWalk::fetchNextBatch(Handle, long, long, int, int, objArrayHandle, Thread*)+0xd9
V  [libjvm.dylib+0x417264]  JVM_MoreStackWalk+0x12f
j  java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(JJII[Ljava/lang/Object;)I+0 java.base@11.0.3
j  java.lang.StackStreamFactory$AbstractStackWalker.fetchStackFrames(I)I+35 java.base@11.0.3
j  java.lang.StackStreamFactory$AbstractStackWalker.getNextBatch()I+61 java.base@11.0.3
j  java.lang.StackStreamFactory$AbstractStackWalker.peekFrame()Ljava/lang/Class;+32 java.base@11.0.3
j  java.lang.StackStreamFactory$AbstractStackWalker.hasNext()Z+1 java.base@11.0.3
j  java.lang.StackStreamFactory$StackFrameTraverser.nextStackFrame()Ljava/lang/StackWalker$StackFrame;+1 java.base@11.0.3
j  java.lang.StackStreamFactory$StackFrameTraverser.forEachRemaining(Ljava/util/function/Consumer;)V+18 java.base@11.0.3
j  java.util.stream.ReferencePipeline$Head.forEach(Ljava/util/function/Consumer;)V+12 java.base@11.0.3
j  java.lang.StackWalker.lambda$forEach$0(Ljava/util/function/Consumer;Ljava/util/stream/Stream;)Ljava/lang/Object;+2 java.base@11.0.3
j  java.lang.StackWalker$$Lambda$291.apply(Ljava/lang/Object;)Ljava/lang/Object;+8 java.base@11.0.3
j  java.lang.StackStreamFactory$StackFrameTraverser.consumeFrames()Ljava/lang/Object;+25 java.base@11.0.3
j  java.lang.StackStreamFactory$AbstractStackWalker.doStackWalk(JIIII)Ljava/lang/Object;+78 java.base@11.0.3
v  ~StubRoutines::call_stub
V  [libjvm.dylib+0x3a3008]  JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x220
V  [libjvm.dylib+0x6905e3]  StackWalk::fetchFirstBatch(BaseFrameStream&, Handle, long, int, int, int, objArrayHandle, Thread*)+0x40d
V  [libjvm.dylib+0x6901b4]  StackWalk::walk(Handle, long, int, int, int, objArrayHandle, Thread*)+0x128
V  [libjvm.dylib+0x417118]  JVM_CallStackWalk+0x1c9
j  java.lang.StackStreamFactory$AbstractStackWalker.callStackWalk(JIII[Ljava/lang/Object;)Ljava/lang/Object;+0 java.base@11.0.3
j  java.lang.StackStreamFactory$AbstractStackWalker.beginStackWalk()Ljava/lang/Object;+31 java.base@11.0.3
j  java.lang.StackStreamFactory$AbstractStackWalker.walk()Ljava/lang/Object;+8 java.base@11.0.3
j  java.lang.StackWalker.forEach(Ljava/util/function/Consumer;)V+15 java.base@11.0.3
v  ~StubRoutines::call_stub
V  [libjvm.dylib+0x3a3008]  JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x220
V  [libjvm.dylib+0x65230d]  invoke(InstanceKlass*, methodHandle const&, Handle, bool, objArrayHandle, BasicType, objArrayHandle, bool, Thread*)+0x6c8
V  [libjvm.dylib+0x651bfb]  Reflection::invoke_method(oopDesc*, Handle, objArrayHandle, Thread*)+0xf7
V  [libjvm.dylib+0x423cc5]  JVM_InvokeMethod+0x1aa
J 6687  jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; java.base@11.0.3 (0 bytes) @ 0x0000000118064ec0 [0x0000000118064dc0+0x0000000000000100]
J 6686 c1 jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; java.base@11.0.3 (104 bytes) @ 0x00000001116ae494 [0x00000001116ae0c0+0x00000000000003d4]
J 5259 c1 jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; java.base@11.0.3 (10 bytes) @ 0x0000000117f6fa04 [0x0000000117f6f9c0+0x0000000000000044]
J 5216 c1 java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; java.base@11.0.3 (65 bytes) @ 0x00000001109f662c [0x00000001109f6500+0x000000000000012c]
j  nonapi.io.github.classgraph.classpath.CallStackReader.getCallStackViaStackWalker()[Ljava/lang/Class;+229
j  nonapi.io.github.classgraph.classpath.CallStackReader.access$100()[Ljava/lang/Class;+4
j  nonapi.io.github.classgraph.classpath.CallStackReader$2.run()[Ljava/lang/Class;+4
j  nonapi.io.github.classgraph.classpath.CallStackReader$2.run()Ljava/lang/Object;+5
v  ~StubRoutines::call_stub
V  [libjvm.dylib+0x3a3008]  JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, Thread*)+0x220
V  [libjvm.dylib+0x41a042]  JVM_DoPrivileged+0x643
j  java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;)Ljava/lang/Object;+0 java.base@11.0.3
j  nonapi.io.github.classgraph.classpath.CallStackReader.getClassContext(Lnonapi/io/github/classgraph/utils/LogNode;)[Ljava/lang/Class;+34
j  nonapi.io.github.classgraph.classpath.ClassLoaderAndModuleFinder.<init>(Lnonapi/io/github/classgraph/ScanSpec;Lnonapi/io/github/classgraph/utils/LogNode;)V+158
j  nonapi.io.github.classgraph.classpath.ClasspathFinder.<init>(Lnonapi/io/github/classgraph/ScanSpec;Lnonapi/io/github/classgraph/utils/LogNode;)V+124
j  io.github.classgraph.Scanner.<init>(Lnonapi/io/github/classgraph/ScanSpec;Ljava/util/concurrent/ExecutorService;ILio/github/classgraph/ClassGraph$ScanResultProcessor;Lio/github/classgraph/ClassGraph$FailureHandler;Lnonapi/io/github/classgraph/utils/LogNode;)V+296
j  io.github.classgraph.ClassGraph.scanAsync(Ljava/util/concurrent/ExecutorService;I)Ljava/util/concurrent/Future;+148
j  io.github.classgraph.ClassGraph.scan(Ljava/util/concurrent/ExecutorService;I)Lio/github/classgraph/ScanResult;+132
j  io.github.classgraph.ClassGraph.scan(I)Lio/github/classgraph/ScanResult;+129
j  io.github.classgraph.ClassGraph.scan()Lio/github/classgraph/ScanResult;+87
j  com.example.wiremock.ClasspathScanningFileSource.getScannedResources()Ljava/util/List;+26
j  com.example.wiremock.ClasspathScanningFileSource.getTextFileNamed(Ljava/lang/String;)Lcom/github/tomakehurst/wiremock/common/TextFile;+5
j  com.example.wiremock.ClasspathScanningFileSourceTest.itShouldLoadFileAtRoot()V+20

[shipilev]

@lukehutch JDK-8221437 is in 13; if backported, it would definitely end up in 11.0.4+ or 12.0.2+. But that’s the assert fix, it should not affect release builds. So, your call if you want to disable StackWalker for 11.0.4 (I wouldn’t).

I took jdk-updates/jdk11u-dev forest (which is bleeding edge 11.0.4+), and ran the reproducer from @bertramn:

1. reverted JDK-8210457: got SEGV in ResolvedMethodTable::add_method(Handle)+0x1cf
2. re-applied JDK-8210457: got assert(java_lang_invoke_ResolvedMethodName::vmtarget(resolved_method()) == m()) failed: Should not change after link resolution
3. applied JDK-8221437: all pass

Which, I think, confirms JDK-8210457 is the fix, and JDK-8221437 fixes the assert on top 😃

[shipilev]

Very nice, proper assert. The beautiful part for having fastdebug asserts is that you can search their messages in bug tracker. If you do, then you’ll find JDK-8221437, which says this is just a too strong assert. If I apply that patch to current jdk11u-dev tree, it passes your tests. Let me pick that patch to 11u backports.

You might want to check everything with the latest jdk-jdk (https://builds.shipilev.net/openjdk-jdk/) too.