Spark version compatibility and compilation error for Spark 2.0.1
See original GitHub issueHi all,
I’m trying compile latest master branch with spark 2.0.1 and hadoop 2.7.3 and I’m getting the following error:
[error] impossible to get artifacts when data has not been loaded. IvyNode = com.fasterxml.jackson.core#jackson-databind;2.5.4
java.lang.IllegalStateException: impossible to get artifacts when data has not been loaded. IvyNode = com.fasterxml.jackson.core#jackson-databind;2.5.4
Setting hadoop 2.4.0 and spark 1.6.1. compiles fine though. Could someone point me to documentation stating which spark and hadoop versions are supported? Should I manually set the dependencies and if so how?
Thanks!
Issue Analytics
- State:
- Created 7 years ago
- Comments:6 (3 by maintainers)
Top Results From Across the Web
Versioning Policy | Apache Spark
Currently, does not guarantee that a compiled application that is linked against version A will link cleanly against version B without re-compiling. Link-level ......
Read more >Compatibility issue with Scala and Spark for compiled jars
The spark version can be different that the installed one as spark has backward compatibility (at least in major version) so you can...
Read more >Spark 2.0.1-1703 Release Notes
Spark-SQL is supported, but it is not fully compatible with Hive. For details, see the Apache Spark documentation and the MapR Spark documentation....
Read more >Elasticsearch-spark not compatible with Spark 2.0.0
spark which is not available. ... [error] the classpath might be incompatible with the version used when compiling Column.class. ... [error] in ...
Read more >Databricks Runtime 10.5 (Unsupported) - Azure
[SPARK-38107] [SQL] Use error classes in the compilation errors of python/pandas UDFs; [SPARK-38354] [SQL] Add hash probes metric for shuffled ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I solve the problem by modify
sbt.version
to0.13.9
inproject/build.properties
, no need to change jacksonVersion to 2.5.4.+1 hadoop 2.7.3/spark 2.1.0 (HDP 2.6) even with sbt 0.13.9 could not complete compilation for me