Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Spark version compatibility and compilation error for Spark 2.0.1

See original GitHub issue

Hi all,

I’m trying compile latest master branch with spark 2.0.1 and hadoop 2.7.3 and I’m getting the following error:

[error] impossible to get artifacts when data has not been loaded. IvyNode = com.fasterxml.jackson.core#jackson-databind;2.5.4
java.lang.IllegalStateException: impossible to get artifacts when data has not been loaded. IvyNode = com.fasterxml.jackson.core#jackson-databind;2.5.4

Setting hadoop 2.4.0 and spark 1.6.1. compiles fine though. Could someone point me to documentation stating which spark and hadoop versions are supported? Should I manually set the dependencies and if so how?

Thanks!

Issue Analytics

State:
Created 7 years ago
Comments:6 (3 by maintainers)

Top GitHub Comments

3reactions

suiyuan2009commented, Feb 28, 2017

I solve the problem by modify sbt.version to 0.13.9 in project/build.properties, no need to change jacksonVersion to 2.5.4.

0reactions

xor007commented, Aug 31, 2017

+1 hadoop 2.7.3/spark 2.1.0 (HDP 2.6) even with sbt 0.13.9 could not complete compilation for me

Top Results From Across the Web

Versioning Policy | Apache Spark

Currently, does not guarantee that a compiled application that is linked against version A will link cleanly against version B without re-compiling. Link-level ......

Compatibility issue with Scala and Spark for compiled jars

The spark version can be different that the installed one as spark has backward compatibility (at least in major version) so you can...

Spark 2.0.1-1703 Release Notes

Spark-SQL is supported, but it is not fully compatible with Hive. For details, see the Apache Spark documentation and the MapR Spark documentation....

Elasticsearch-spark not compatible with Spark 2.0.0

spark which is not available. ... [error] the classpath might be incompatible with the version used when compiling Column.class. ... [error] in ...

Databricks Runtime 10.5 (Unsupported) - Azure

[SPARK-38107] [SQL] Use error classes in the compilation errors of python/pandas UDFs; [SPARK-38354] [SQL] Add hash probes metric for shuffled ...