Setting JEP native library path for java >10

I am opening a specific issue for this, although it was mentioned #603.

The JEP instance will load itself the native JEP library, but for this to happen we need to add the path to this native library in the java.library.path. So far it was done dynamically, via the user usr_paths field where we appended the JEP native path, and only if some Deep Learning model was required by the config. Unfortunately usr_paths could not be accessed any more as Field via the ClassLoader from Java 11 for security reason.

A possible solution is to use MethodHandler as explained here but it seems not working then for java <9 and stops working again for java 15 because usr_paths cannot be accessed via the ClassLoader any more at all (Caused by: java.lang.NoSuchFieldError: usr_paths).

So adding new paths for native libraries at runtime in Java seems a dead end.

The alternative would be to add the JEP native path when launching the JDK via Gradle, with something like this:

    systemProperty "java.library.path", file("grobid-home/lib/**os-arch***/").absolutePath

If JEP is not used, the library won’t be loaded and this path won’t be used anyway, so it’s not a problem to add the path every time at launch.

For this, we need to identify the right OS info from Gradle (see maybe here for an example), but also normally the version of the JDK and the version of python installed locally…

Note: It does not affect the docker image which will always work fine. In the case of the Docker image, the right JEP is in the system library path because it is fully installed at system-level on the image, and we are not using the embedded native JEP library at all.

[kermitt2]

Did you try the https://github.com/kermitt2/grobid/issues/688#issuecomment-1140774115 that should extend the list of paths? it does not work?

Sorry, I’ve tried and it works ! I just had to include also the path to grobid-home/lib/lin-64 for other libraries.

systemProperty "java.library.path","${System.getProperty('java.library.path')}:${file("../grobid-home/lib/lin-64/jep").absolutePath}:${file("../grobid-home/lib/lin-64").absolutePath}"

🙌 ✨.

This is great, now we normally only need to make it OS specific and apply it to the run task.

[kermitt2]

I started a PR for this https://github.com/kermitt2/grobid/pull/921