Please consider ensuring compatibility with JimFs

Please consider ensuring compatibility with JimFs (in-memory file system). (Or document its use?)

I can’t make classgraph scan an in-memory file system. Here is an example code, using an empty in-memory file system. Dependent on com.google.jimfs:jimfs:1.1. Expected: does nothing. Actual: crashes with the stack trace printed below.

try (FileSystem memFs = Jimfs.newFileSystem()) {
	Path pathInMem = memFs.getPath("");

	URL url = pathInMem.toUri().toURL();
	try (URLClassLoader child = new URLClassLoader(new URL[] { url }, getClass().getClassLoader())) {
		ClassGraph graph = new ClassGraph().enableURLScheme(url.getProtocol()).overrideClassLoaders(child)
					.ignoreParentClassLoaders().enableAllInfo();
		try (ScanResult scanResult = graph.verbose().scan()) {
			// FAILS!
		}
	}
}

avr. 23, 2020 12:00:31 AM nonapi.io.github.classgraph.utils.LogNode flush INFOS: 2020-04-23T00:00:31.244+0200 ClassGraph ClassGraph version 4.8.72 2020-04-23T00:00:31.244+0200 ClassGraph Operating system: Linux 4.19.0-8-amd64 amd64 2020-04-23T00:00:31.245+0200 ClassGraph Java version: 11.0.6 / 11.0.6+10-post-Debian-1deb10u1 (Debian) 2020-04-23T00:00:31.245+0200 ClassGraph Java home: /usr/lib/jvm/java-11-openjdk-amd64 2020-04-23T00:00:31.247+0200 ClassGraph ScanSpec: 2020-04-23T00:00:31.248+0200 ClassGraph – packageWhiteBlackList: 2020-04-23T00:00:31.248+0200 ClassGraph – packagePrefixWhiteBlackList: 2020-04-23T00:00:31.248+0200 ClassGraph – pathWhiteBlackList: 2020-04-23T00:00:31.248+0200 ClassGraph – pathPrefixWhiteBlackList: 2020-04-23T00:00:31.248+0200 ClassGraph – classWhiteBlackList: 2020-04-23T00:00:31.262+0200 ClassGraph – classfilePathWhiteBlackList: 2020-04-23T00:00:31.262+0200 ClassGraph – classPackageWhiteBlackList: 2020-04-23T00:00:31.262+0200 ClassGraph – classPackagePathWhiteBlackList: 2020-04-23T00:00:31.262+0200 ClassGraph – moduleWhiteBlackList: 2020-04-23T00:00:31.262+0200 ClassGraph – jarWhiteBlackList: 2020-04-23T00:00:31.263+0200 ClassGraph – classpathElementResourcePathWhiteBlackList: 2020-04-23T00:00:31.263+0200 ClassGraph – libOrExtJarWhiteBlackList: 2020-04-23T00:00:31.263+0200 ClassGraph – scanJars: true 2020-04-23T00:00:31.263+0200 ClassGraph – scanNestedJars: true 2020-04-23T00:00:31.263+0200 ClassGraph – scanDirs: true 2020-04-23T00:00:31.263+0200 ClassGraph – scanModules: true 2020-04-23T00:00:31.264+0200 ClassGraph – enableClassInfo: true 2020-04-23T00:00:31.264+0200 ClassGraph – enableFieldInfo: true 2020-04-23T00:00:31.264+0200 ClassGraph – enableMethodInfo: true 2020-04-23T00:00:31.264+0200 ClassGraph – enableAnnotationInfo: true 2020-04-23T00:00:31.264+0200 ClassGraph – enableStaticFinalFieldConstantInitializerValues: true 2020-04-23T00:00:31.264+0200 ClassGraph – enableInterClassDependencies: false 2020-04-23T00:00:31.264+0200 ClassGraph – enableExternalClasses: false 2020-04-23T00:00:31.264+0200 ClassGraph – enableSystemJarsAndModules: false 2020-04-23T00:00:31.264+0200 ClassGraph – ignoreClassVisibility: true 2020-04-23T00:00:31.265+0200 ClassGraph – ignoreFieldVisibility: true 2020-04-23T00:00:31.265+0200 ClassGraph – ignoreMethodVisibility: true 2020-04-23T00:00:31.265+0200 ClassGraph – disableRuntimeInvisibleAnnotations: false 2020-04-23T00:00:31.265+0200 ClassGraph – extendScanningUpwardsToExternalClasses: true 2020-04-23T00:00:31.265+0200 ClassGraph – allowedURLSchemes: [jimfs] 2020-04-23T00:00:31.265+0200 ClassGraph – addedClassLoaders: null 2020-04-23T00:00:31.265+0200 ClassGraph – overrideClassLoaders: [java.net.URLClassLoader@30af5b6b] 2020-04-23T00:00:31.266+0200 ClassGraph – addedModuleLayers: null 2020-04-23T00:00:31.266+0200 ClassGraph – overrideModuleLayers: null 2020-04-23T00:00:31.266+0200 ClassGraph – overrideClasspath: null 2020-04-23T00:00:31.266+0200 ClassGraph – classpathElementFilters: null 2020-04-23T00:00:31.266+0200 ClassGraph – initializeLoadedClasses: false 2020-04-23T00:00:31.266+0200 ClassGraph – removeTemporaryFilesAfterScan: false 2020-04-23T00:00:31.266+0200 ClassGraph – ignoreParentClassLoaders: true 2020-04-23T00:00:31.267+0200 ClassGraph – ignoreParentModuleLayers: false 2020-04-23T00:00:31.267+0200 ClassGraph – modulePathInfo: 2020-04-23T00:00:31.267+0200 ClassGraph – maxBufferedJarRAMSize: 67108864 2020-04-23T00:00:31.267+0200 ClassGraph – enableMemoryMapping: false 2020-04-23T00:00:31.267+0200 ClassGraph Number of worker threads: 4 2020-04-23T00:00:31.267+0200 ClassGraph Finding classpath 2020-04-23T00:00:31.267+0200 ClassGraph – Finding classpath and modules 2020-04-23T00:00:31.268+0200 ClassGraph ---- ClassLoaderHandlers: 2020-04-23T00:00:31.268+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.AntClassLoaderHandler 2020-04-23T00:00:31.268+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.EquinoxClassLoaderHandler 2020-04-23T00:00:31.268+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.EquinoxContextFinderClassLoaderHandler 2020-04-23T00:00:31.268+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.FelixClassLoaderHandler 2020-04-23T00:00:31.268+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.JBossClassLoaderHandler 2020-04-23T00:00:31.268+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.WeblogicClassLoaderHandler 2020-04-23T00:00:31.268+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.WebsphereLibertyClassLoaderHandler 2020-04-23T00:00:31.268+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.WebsphereTraditionalClassLoaderHandler 2020-04-23T00:00:31.268+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.OSGiDefaultClassLoaderHandler 2020-04-23T00:00:31.269+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.SpringBootRestartClassLoaderHandler 2020-04-23T00:00:31.269+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.TomcatWebappClassLoaderBaseHandler 2020-04-23T00:00:31.269+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.PlexusClassWorldsClassRealmClassLoaderHandler 2020-04-23T00:00:31.269+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.QuarkusClassLoaderHandler 2020-04-23T00:00:31.269+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.ParentLastDelegationOrderTestClassLoaderHandler 2020-04-23T00:00:31.278+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.JPMSClassLoaderHandler 2020-04-23T00:00:31.278+0200 ClassGraph ------ nonapi.io.github.classgraph.classloaderhandler.URLClassLoaderHandler 2020-04-23T00:00:31.278+0200 ClassGraph ---- Finding unique classloaders in delegation order 2020-04-23T00:00:31.279+0200 ClassGraph ---- Obtaining URLs from classloaders in delegation order 2020-04-23T00:00:31.280+0200 ClassGraph ------ Ignoring parent classloader jdk.internal.loader.ClassLoaders$PlatformClassLoader@1f760b47, normally handled by nonapi.io.github.classgraph.classloaderhandler.JPMSClassLoaderHandler 2020-04-23T00:00:31.281+0200 ClassGraph ------ Ignoring parent classloader jdk.internal.loader.ClassLoaders$AppClassLoader@5ffd2b27, normally handled by nonapi.io.github.classgraph.classloaderhandler.JPMSClassLoaderHandler 2020-04-23T00:00:31.281+0200 ClassGraph ------ Classloader java.net.URLClassLoader@30af5b6b is handled by nonapi.io.github.classgraph.classloaderhandler.URLClassLoaderHandler 2020-04-23T00:00:31.281+0200 ClassGraph -------- Found classpath element: jimfs://4dbef289-e533-4340-8531-fe2cc6b2f0f3/work/ 2020-04-23T00:00:31.290+0200 ClassGraph Opening classpath elements (took 0,088292 sec) 2020-04-23T00:00:31.351+0200 ClassGraph – Opening jar: jimfs://4dbef289-e533-4340-8531-fe2cc6b2f0f3/work/ 2020-04-23T00:00:31.353+0200 ClassGraph ---- Downloading jar from URL jimfs:/4dbef289-e533-4340-8531-fe2cc6b2f0f3/work 2020-04-23T00:00:31.378+0200 ClassGraph – Worker thread was interrupted 2020-04-23T00:00:31.378+0200 ClassGraph Uncaught exception during scan 2020-04-23T00:00:31.379+0200 ClassGraph – java.lang.AssertionError: java.net.URISyntaxException: Expected scheme-specific part at index 6: jimfs: 2020-04-23T00:00:31.379+0200 ClassGraph – at com.google.common.jimfs.SystemJimfsFileSystemProvider.toFileSystemUri(SystemJimfsFileSystemProvider.java:166) 2020-04-23T00:00:31.379+0200 ClassGraph – at com.google.common.jimfs.SystemJimfsFileSystemProvider.getPath(SystemJimfsFileSystemProvider.java:144) 2020-04-23T00:00:31.379+0200 ClassGraph – at java.base/java.nio.file.Path.of(Path.java:208) 2020-04-23T00:00:31.379+0200 ClassGraph – at java.base/java.nio.file.Paths.get(Paths.java:97) 2020-04-23T00:00:31.379+0200 ClassGraph – at com.google.common.jimfs.PathURLConnection.connect(PathURLConnection.java:76) 2020-04-23T00:00:31.379+0200 ClassGraph – at com.google.common.jimfs.PathURLConnection.getInputStream(PathURLConnection.java:122) 2020-04-23T00:00:31.379+0200 ClassGraph – at nonapi.io.github.classgraph.fastzipfilereader.NestedJarHandler.downloadJarFromURL(NestedJarHandler.java:541) 2020-04-23T00:00:31.379+0200 ClassGraph – at nonapi.io.github.classgraph.fastzipfilereader.NestedJarHandler.access$000(NestedJarHandler.java:77) 2020-04-23T00:00:31.379+0200 ClassGraph – at nonapi.io.github.classgraph.fastzipfilereader.NestedJarHandler$4.newInstance(NestedJarHandler.java:179) 2020-04-23T00:00:31.379+0200 ClassGraph – at nonapi.io.github.classgraph.fastzipfilereader.NestedJarHandler$4.newInstance(NestedJarHandler.java:150) 2020-04-23T00:00:31.379+0200 ClassGraph – at nonapi.io.github.classgraph.concurrency.SingletonMap.get(SingletonMap.java:189) 2020-04-23T00:00:31.379+0200 ClassGraph – at io.github.classgraph.ClasspathElementZip.open(ClasspathElementZip.java:140) 2020-04-23T00:00:31.379+0200 ClassGraph – at io.github.classgraph.Scanner$3.processWorkUnit(Scanner.java:528) 2020-04-23T00:00:31.379+0200 ClassGraph – at io.github.classgraph.Scanner$3.processWorkUnit(Scanner.java:503) 2020-04-23T00:00:31.379+0200 ClassGraph – at nonapi.io.github.classgraph.concurrency.WorkQueue.runWorkLoop(WorkQueue.java:246) 2020-04-23T00:00:31.379+0200 ClassGraph – at nonapi.io.github.classgraph.concurrency.WorkQueue.access$000(WorkQueue.java:50) 2020-04-23T00:00:31.379+0200 ClassGraph – at nonapi.io.github.classgraph.concurrency.WorkQueue$1.call(WorkQueue.java:201) 2020-04-23T00:00:31.379+0200 ClassGraph – at nonapi.io.github.classgraph.concurrency.WorkQueue$1.call(WorkQueue.java:198) 2020-04-23T00:00:31.379+0200 ClassGraph – at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) 2020-04-23T00:00:31.379+0200 ClassGraph – at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) 2020-04-23T00:00:31.379+0200 ClassGraph – at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) 2020-04-23T00:00:31.379+0200 ClassGraph – at java.base/java.lang.Thread.run(Thread.java:834) 2020-04-23T00:00:31.379+0200 ClassGraph – Caused by: java.net.URISyntaxException: Expected scheme-specific part at index 6: jimfs: 2020-04-23T00:00:31.379+0200 ClassGraph – at java.base/java.net.URI$Parser.fail(URI.java:2913) 2020-04-23T00:00:31.379+0200 ClassGraph – at java.base/java.net.URI$Parser.failExpecting(URI.java:2919) 2020-04-23T00:00:31.379+0200 ClassGraph – at java.base/java.net.URI$Parser.parse(URI.java:3119) 2020-04-23T00:00:31.379+0200 ClassGraph – at java.base/java.net.URI.<init>(URI.java:685) 2020-04-23T00:00:31.379+0200 ClassGraph – at com.google.common.jimfs.SystemJimfsFileSystemProvider.toFileSystemUri(SystemJimfsFileSystemProvider.java:163) 2020-04-23T00:00:31.379+0200 ClassGraph – … 21 more

[lukehutch]

OK, I added support to ClassGraph for scanning Path objects in custom filesystems, whether they are directories or files. Therefore you can now specify the Jimfs path of a directory, and the recursive scanning will use the Path API rather than the File API. If you specify the Jimfs path of a jarfile, it will be opened as a jarfile using FileChannel.

This should work for any custom URI scheme that is backed by an underlying FileSystem implementation. If a custom URI scheme is not backed by an underlying FileSystem implementation, then URL.openConnection() is used instead.

Released in 4.8.75. Thanks for the suggestion!

[lukehutch]

@oliviercailloux Fixed and released in 4.8.76. There seems to be a Windows regression though, so that will be fixed in 4.8.77 once I can figure out what is causing the issue.