NoClassDefFoundError encountered when using Producer (Java API)

Expected behavior

I wrote a simple Scala program to publish a single message to a local standalone Pulsar cluster. I expected the message to be successfully sent. Here is the Scala program:

package main

import org.apache.pulsar.client.api._

object Main {
  val localClusterUrl = "pulsar://localhost:6650"
  val topicName = "persistent://sample/standalone/ns1/my-topic"

  def main(args: Array[String]): Unit = {
    val client = PulsarClient.create(localClusterUrl)
    val producer = client.createProducer(topicName)
    producer.send("message".getBytes)
    producer.close()
    client.close()
  }
}

And the build.sbt:

mainClass := Some("Main")

libraryDependencies ++= {
    Seq(
      "org.apache.pulsar" % "pulsar-client" % "1.20.0-incubating"
    )
}

Actual behavior

When I run the above program, the following error is output:

[error] (run-main-0) org.apache.pulsar.client.api.PulsarClientException: java.lang.NoClassDefFoundError: org/apache/pulsar/checksum/utils/Crc32cChecksum
org.apache.pulsar.client.api.PulsarClientException: java.lang.NoClassDefFoundError: org/apache/pulsar/checksum/utils/Crc32cChecksum
	at org.apache.pulsar.client.impl.ProducerImpl.sendAsync(ProducerImpl.java:342)
	at org.apache.pulsar.client.impl.ProducerImpl.sendAsync(ProducerImpl.java:182)
	at org.apache.pulsar.client.impl.ProducerBase.send(ProducerBase.java:57)
	at org.apache.pulsar.client.impl.ProducerBase.send(ProducerBase.java:46)
	at main.Main$.main(main.scala:16)
	at main.Main.main(main.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
Caused by: java.lang.NoClassDefFoundError: org/apache/pulsar/checksum/utils/Crc32cChecksum
	at org.apache.pulsar.common.api.Commands.serializeCommandSendWithSize(Commands.java:686)
	at org.apache.pulsar.common.api.Commands.newSend(Commands.java:298)
	at org.apache.pulsar.client.impl.ProducerImpl.sendMessage(ProducerImpl.java:376)
	at org.apache.pulsar.client.impl.ProducerImpl.sendAsync(ProducerImpl.java:307)
	at org.apache.pulsar.client.impl.ProducerImpl.sendAsync(ProducerImpl.java:182)
	at org.apache.pulsar.client.impl.ProducerBase.send(ProducerBase.java:57)
	at org.apache.pulsar.client.impl.ProducerBase.send(ProducerBase.java:46)
	at main.Main$.main(main.scala:16)
	at main.Main.main(main.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
Caused by: java.lang.ClassNotFoundException: org.apache.pulsar.checksum.utils.Crc32cChecksum
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at org.apache.pulsar.common.api.Commands.serializeCommandSendWithSize(Commands.java:686)
	at org.apache.pulsar.common.api.Commands.newSend(Commands.java:298)
	at org.apache.pulsar.client.impl.ProducerImpl.sendMessage(ProducerImpl.java:376)
	at org.apache.pulsar.client.impl.ProducerImpl.sendAsync(ProducerImpl.java:307)
	at org.apache.pulsar.client.impl.ProducerImpl.sendAsync(ProducerImpl.java:182)
	at org.apache.pulsar.client.impl.ProducerBase.send(ProducerBase.java:57)
	at org.apache.pulsar.client.impl.ProducerBase.send(ProducerBase.java:46)
	at main.Main$.main(main.scala:16)
	at main.Main.main(main.scala)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)

This is unexpected. I would appreciate any ideas as to what the issue could be.

Steps to reproduce

Install sbt. Create a new directory/working space. Create build.sbt as shown above. Create src/main/scala/main.scala as shown above. Run “sbt”. In the sbt prompt, execute “compile” and “run”.

System configuration

“org.apache.pulsar” % “pulsar-client” % “1.20.0-incubating” sbt 0.13.15 java 1.8.0_144 Scala 2.11.2

[rstout610]

Thanks @merlimat, the “shaded” classifier worked!

[sijie]

@otmanel31 how do you build your NIFI processor? what are the dependencies you are using?