DynamoDb-enhanced NoClassDefFoundError with different ClassLoaders

The BeanTableSchema of dynamodb-enhanced does not work correctly when this library and the bean class reside in different classloaders. A NoClassDefFoundError for the bean class is thrown, when a call against a bean class/object is made (<init>, getter, setter).

Describe the bug

Internally, the BeanTableSchema creates “Accessor”-Lambdas for performant creation of bean objects, as well as getters and setters for the properties. An example would be the software.amazon.awssdk.enhanced.dynamodb.mapper.BeanTableSchema#newObjectSupplierForClass (also #getterForProperty and #setterForProperty) responsible for creating a Supplier that returns a new bean instance.

Internally, the LambdaToMethodBridgeBuilder is then used to construct such lambda via the LambdaMetafactory. The creation of the lambda itself is successful.

However, when this generated lambda is invoked, it crashes with the mentioned NoClassDefFoundError. This is related to the fact that the Lambda (somehow) tries to resolve the bean class with the classloader of the SDK. But since the bean class is loaded in a different classloader, the classloader of the SDK cannot find the bean class. More specifically, the problem arises when the SDK classloader is either a parent of or entirely unrelated to the classloader of the bean.

This issue is most likely related to issue #2198 since the error messages are identical. However that issue is specifically about usage with the play framework (which I do not know anything about). Furthermore, a usage of Class.forName is mentioned but I did not come across usage of that. Therefore I decided to create a more focused issue for the core problem I (somewhat) have a solution for.

This issue should fix the NoClassDefFoundError of https://github.com/quarkusio/quarkus/issues/12168.

Expected Behavior

Normal execution and same behavior when AWS SDK and bean class are in the same classloader.

Current Behavior

When such generated lambda is invoked, the following (sample) stack trace is thrown:

Caused by: java.lang.NoClassDefFoundError: org/acme/dynamodb/Fruit
	at software.amazon.awssdk.enhanced.dynamodb.internal.mapper.ResolvedImmutableAttribute.lambda$create$0(ResolvedImmutableAttribute.java:48)
	at software.amazon.awssdk.enhanced.dynamodb.mapper.StaticImmutableTableSchema.lambda$itemToMap$5(StaticImmutableTableSchema.java:502)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
	at java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1085)
	at software.amazon.awssdk.enhanced.dynamodb.mapper.StaticImmutableTableSchema.itemToMap(StaticImmutableTableSchema.java:500)
	at software.amazon.awssdk.enhanced.dynamodb.mapper.WrappedTableSchema.itemToMap(WrappedTableSchema.java:64)
	at software.amazon.awssdk.enhanced.dynamodb.mapper.WrappedTableSchema.itemToMap(WrappedTableSchema.java:64)
	at software.amazon.awssdk.enhanced.dynamodb.internal.operations.PutItemOperation.generateRequest(PutItemOperation.java:71)
	at software.amazon.awssdk.enhanced.dynamodb.internal.operations.PutItemOperation.generateRequest(PutItemOperation.java:40)
	at software.amazon.awssdk.enhanced.dynamodb.internal.operations.CommonOperation.execute(CommonOperation.java:113)
	at software.amazon.awssdk.enhanced.dynamodb.internal.operations.TableOperation.executeOnPrimaryIndex(TableOperation.java:59)
	at software.amazon.awssdk.enhanced.dynamodb.internal.client.DefaultDynamoDbTable.putItem(DefaultDynamoDbTable.java:180)
	at software.amazon.awssdk.enhanced.dynamodb.internal.client.DefaultDynamoDbTable.putItem(DefaultDynamoDbTable.java:188)
	at software.amazon.awssdk.enhanced.dynamodb.internal.client.DefaultDynamoDbTable.putItem(DefaultDynamoDbTable.java:193)
	at org.acme.dynamodb.FruitSyncService.add(FruitSyncService.java:49)
	at org.acme.dynamodb.FruitSyncService_ClientProxy.add(FruitSyncService_ClientProxy.zig:354)
	at org.acme.dynamodb.FruitResource.add(FruitResource.java:40)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:170)
	at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:130)
	at org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:660)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:524)
	at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$2(ResourceMethodInvoker.java:474)
	at org.jboss.resteasy.core.interception.jaxrs.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:364)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:476)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:434)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:408)
	at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:69)
	at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:492)
	... 15 more
Caused by: java.lang.ClassNotFoundException: org.acme.dynamodb.Fruit
	at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:421)
	at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:397)
	... 47 more

(where Fruit is the bean class.)

The code that creates the dynamodb client looks like this:

DynamoDbEnhancedClient enhancedClient = DynamoDbEnhancedClient.builder().dynamoDbClient(ddb).build();
var table = enhancedClient.table("Fruits", TableSchema.fromBean(Fruit.class));

offending user-application code is:

table.putItem(fruit);

Steps to Reproduce

Loading a dynamodb-enhanced annotated bean in a child (or unrelated) classloader, pass it to TableSchema.fromBean(...) and use any action like getItem or putItem.

A self-contained runnable example can be found here: https://github.com/Nithanim/awssdk2-dynamodb-enhanced-NDFE/tree/master/src/main/java

You can find a “crashing” example specifically using the problematic AWS SDK class. Additionally there is an isolated case without the AWS SDK only with plain java.

Possible Solution

When creating the Lambdas via the LambdaMetafactory the Lookup of the AWS SDK is used. This is (as far as I understand) the cause that the wrong classloader for the lookup is used.

A working solution would be to use the Lookup of the bean class, which in turn will use the classloader of the bean class to resolve the bean class on invocation of the lambda.

In the same repository (https://github.com/Nithanim/awssdk2-dynamodb-enhanced-NDFE/tree/master/src/main/java) you can also find the “Fixed” classes, which have been patched to reslove the issue. However, I am completely unsure if Lookup.privateLookupIn(...) is the correct way.

Context

I wanted to use the dynamodb-enhanced in quarkus (and specifically later in native-image) but the Tests kept failing with the NoClassDefFoundError. After some digging I found out that quarkus uses two different classloaders for tests. One for all libraries and then a child classloader for the tests. A bit of insight can be found here: https://quarkus.io/guides/class-loading-reference

Thank you for your help!

Your Environment

• AWS Java SDK version used: 2.16.81
• JDK version used: build 11.0.11+9-Ubuntu-0ubuntu2.20.04
• Operating System and version: Ubuntu 20.04.2 LTS

[Nithanim]

I noticed that my proposed solution only works for Java 9+ but this library is 8+. Sadly I have not found any other good solution.

I have not been able to find a workaround just using the MethodHandles/Lookups provided by Java 8 alone. A workaround would be to create child ClassLoader (CL) of the bean CL in which we can load a class from which we can fetch a Lookup for that context.

However, this is rather ugly because we would need the bytecode of such class and then load it in a classloader as shown here. My proof-of-concept with bytebuddy is already convoluted enough considering this is all just a dance-around for the simple Lookup.privateLookupIn() of Java 9+.

In short:

  private static MethodHandles.Lookup getPrivateLookup(ClassLoader beanClassLoader) {
    // We use BB to be able to simply define the class below and load a copy in a child CL
    ByteBuddy bb = new ByteBuddy();
    DynamicType.Unloaded<GetLookup> clone =
        bb.rebase(GetLookup.class).name("JustRenameBecauseClassIsAlreadyLoadedInParentCL").make();
    Class<? extends Supplier<MethodHandles.Lookup>> loaded =
        clone.load(beanClassLoader).getLoaded(); // Creates child CL automatically
    try {
      Supplier<MethodHandles.Lookup> supplier = loaded.getConstructor().newInstance();
      return supplier.get();
    } catch (Exception ex) {
      throw new IllegalStateException(ex);
    }
  }

  public static class GetLookup implements Supplier<MethodHandles.Lookup> {
    public MethodHandles.Lookup get() {
      return MethodHandles.lookup();
    }
  }

[vaibhav170]

Yes right, let me try, my observation was couple of months old, just now i saw another thread where you guys are discussing about fix, will try that and also update on cold start with both the approach, thanks