Building Pulsar (incl. docker images) from source with Java 9+ (Java 11, Java 15, etc.) causes silent NoSuchMethodErrors at runtime

Building Pulsar (incl. docker images) from source with Java 11 causes silent NoSuchMethodErrors at runtime

To Reproduce Steps to reproduce the behavior:

Use Java 11. Then, compile v2.6.2-candidate-1 docker images locally:

git clone --depth 1 -b v2.6.2-candidate-1 https://github.com/apache/pulsar pulsar-2.6
cd pulsar-2.6
mvn -DskipTests clean install
./docker/build.sh

Start pulsar in docker

docker run -p 6650:6650 pulsar:2.6.2 bin/pulsar standalone --no-functions-worker -nss

Compile & run this Pulsar code with required libraries:

import org.apache.pulsar.client.api.*;

public class PulsarJava11Problem {
    public static void main(String[] args) throws PulsarClientException {
        PulsarClient pulsarClient = PulsarClient.builder().serviceUrl("pulsar://localhost:6650").build();
        Producer<String> producer = pulsarClient.newProducer(Schema.STRING).topic("mytopic").create();
    }
}

The producer never gets created and it times out.

What happens on a server is a java.lang.NoSuchMethodError: java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;

This happens at https://github.com/apache/pulsar/blob/097108a7abae750a239abb606d2a31442319e741/pulsar-common/src/main/java/org/apache/pulsar/common/schema/LongSchemaVersion.java#L44 .

java.lang.NoSuchMethodError: java.nio.ByteBuffer.rewind()Ljava/nio/ByteBuffer;
	at org.apache.pulsar.common.schema.LongSchemaVersion.bytes(LongSchemaVersion.java:44)
	at org.apache.pulsar.common.protocol.Commands.newProducerSuccessCommand(Commands.java:377)
	at org.apache.pulsar.broker.service.PulsarCommandSenderImpl.sendProducerSuccessResponse(PulsarCommandSenderImpl.java:95)
	at org.apache.pulsar.broker.service.ServerCnx.lambda$handleProducer$19(ServerCnx.java:1137)
	at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670)
	at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:646)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
	at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:670)
	at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:646)
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
	at org.apache.pulsar.broker.service.schema.BookkeeperSchemaStorage.lambda$createSchemaLocator$28(BookkeeperSchemaStorage.java:504)
	at org.apache.bookkeeper.util.ZkUtils$1.processResult(ZkUtils.java:80)
	at org.apache.bookkeeper.zookeeper.ZooKeeperClient$10$1.processResult(ZooKeeperClient.java:737)
	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:636)
	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510)

This exception never gets printed to the log, this stacktrace was captured using the debugger.

Expected behavior

Compiling with an unsupported Java version should result in an error and the NoSuchMethodError that happens in Pulsar shouldn’t get “swallowed”.

It should be possible to compile the code with Java 9+ without running into issues when running with Java 8.

There’s an explanation of the issue with ByteBuffer at https://github.com/eclipse/jetty.project/issues/3244 . A solution is also described:

The solution is to cast the ByteBuffer to Buffer when calling those methods:

((Buffer)byteBuffer).position(0);

[wolfstudy]

This problem should not only exist in 2.6.2, but it should also be a master problem, @zymap helped to verify the logic with JDK 1.8, it seems that everything is OK. It looks like we need to support different JDK versions to build Apache Pulsar.

[eolivelli]

You have to add it in a profile, because if you add only this property the build will break on jdk8 because the ‘release’ option is not valid with jdk8