Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Issue with Spark/Cassandra/commons-codecs

See original GitHub issue

When using zipkin-dependencies against a non local spark cluster, I get

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.encodeBase64String([B)Ljava/lang/String;
	at com.datastax.spark.connector.cql.CassandraConnectorConf.serializedConfString$lzycompute(CassandraConnectorConf.scala:42)
...
at zipkin.dependencies.cassandra.CassandraDependenciesJob.run(CassandraDependenciesJob.java:149)
	at zipkin.dependencies.ZipkinDependenciesJob.main(ZipkinDependenciesJob.java:48)

Full trace


Exception in thread "main" java.lang.NoSuchMethodError: org.apache.commons.codec.binary.Base64.encodeBase64String([B)Ljava/lang/String;
	at com.datastax.spark.connector.cql.CassandraConnectorConf.serializedConfString$lzycompute(CassandraConnectorConf.scala:42)
	at com.datastax.spark.connector.cql.CassandraConnectorConf.serializedConfString(CassandraConnectorConf.scala:37)
	at com.datastax.spark.connector.cql.CassandraConnectorConf.hashCode(CassandraConnectorConf.scala:45)
	at scala.runtime.ScalaRunTime$.hash(ScalaRunTime.scala:210)
	at scala.collection.concurrent.TrieMap$MangledHashing.hash(TrieMap.scala:916)
	at scala.collection.concurrent.TrieMap.computeHash(TrieMap.scala:826)
	at scala.collection.concurrent.TrieMap.get(TrieMap.scala:841)
	at com.datastax.spark.connector.cql.RefCountedCache.acquire(RefCountedCache.scala:50)
	at com.datastax.spark.connector.cql.CassandraConnector.openSession(CassandraConnector.scala:80)
	at com.datastax.spark.connector.cql.CassandraConnector.withSessionDo(CassandraConnector.scala:107)
	at com.datastax.spark.connector.cql.CassandraConnector.withClusterDo(CassandraConnector.scala:118)
	at com.datastax.spark.connector.cql.Schema$.fromCassandra(Schema.scala:330)
	at com.datastax.spark.connector.cql.Schema$.tableFromCassandra(Schema.scala:350)
	at com.datastax.spark.connector.rdd.CassandraTableRowReaderProvider$class.tableDef(CassandraTableRowReaderProvider.scala:50)
	at com.datastax.spark.connector.rdd.CassandraTableScanRDD.tableDef$lzycompute(CassandraTableScanRDD.scala:60)
	at com.datastax.spark.connector.rdd.CassandraTableScanRDD.tableDef(CassandraTableScanRDD.scala:60)
	at com.datastax.spark.connector.rdd.CassandraTableRowReaderProvider$class.verify(CassandraTableRowReaderProvider.scala:137)
	at com.datastax.spark.connector.rdd.CassandraTableScanRDD.verify(CassandraTableScanRDD.scala:60)
	at com.datastax.spark.connector.rdd.CassandraTableScanRDD.getPartitions(CassandraTableScanRDD.scala:230)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
	at scala.Option.getOrElse(Option.scala:120)
	at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
	at com.datastax.spark.connector.rdd.SpannedRDD.getPartitions(SpannedRDD.scala:25)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
	at scala.Option.getOrElse(Option.scala:120)
	at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
	at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
	at scala.Option.getOrElse(Option.scala:120)
	at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
	at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
	at scala.Option.getOrElse(Option.scala:120)
	at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
	at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
	at scala.Option.getOrElse(Option.scala:120)
	at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
	at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)
	at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)
	at scala.Option.getOrElse(Option.scala:120)
	at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)
	at org.apache.spark.Partitioner$.defaultPartitioner(Partitioner.scala:65)
	at org.apache.spark.api.java.JavaPairRDD.reduceByKey(JavaPairRDD.scala:526)
	at zipkin.dependencies.cassandra.CassandraDependenciesJob.run(CassandraDependenciesJob.java:149)
	at zipkin.dependencies.ZipkinDependenciesJob.main(ZipkinDependenciesJob.java:48)

I have no idea what is happening there. Sounds like some conflicts between projects maybe:(

Any idea?

Issue Analytics

State:
Created 6 years ago
Comments:10 (5 by maintainers)

Top GitHub Comments

1reaction

JodeZercommented, Jun 1, 2017

We encountered the same problem. Hope useful solution can be merged!
😃

0reactions

yscommented, Jun 6, 2017

Works great for us, thanks!

Top Results From Across the Web

Error with spark-cassandra-connector in Spark java.lang ...

Finally resolved using sbt-assembly as suggested by @Odomontois. This is the final build.sbt: name := "Simple Project" version := "1.0" scalaVersion ...

Amazon Keyspaces (for Apache Cassandra) - 亚马逊云科技

Learn how to troubleshoot common errors when using the Spark Cassandra Connector with Amazon Keyspaces.

com.datastax.spark.connector.rdd.partitioner ...

Hi, I installed Spark 1.1.1 from binary. In host:8080 it shows the spark master and two workers. In spark-shell I can run the...

DataStax Spark Connector for Apache Cassandra - Issues

Key Summary Assignee Re... SPARKC‑693 Support for Apache Spark 3.3* Jaroslaw Grabowski Jac... SPARKC‑692 Direct Join ‑ Match Error on aliased columns Jaroslaw Grabowski Ja... SPARKC‑691...

Optional codecs - DataStax Java Driver

<dependency> <groupId>com.datastax.cassandra</groupId> ... Note that if you registered both codecs at the same time, there would be a problem: because ...