Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
Support for Spark 3.x?
See original GitHub issueHi,
Does Stocator work with Spark 3.x? I see there are some dependency conflicts in the versions of Hadoop, etc for Spark 3.x and Stocator. When I create the stocator fat jar and use that with Spark 3.1, I run into Jackson consistency issues, whereas getting a thin jar and using that is missing a lot of dependencies.
Any guidance here will be helpful! Thanks!
Issue Analytics
- State:
- Created 2 years ago
- Comments:5 (1 by maintainers)
Top Results From Across the Web
Overview - Spark 3.0.0 Documentation - Apache Spark
Spark runs on Java 8/11, Scala 2.12, Python 2.7+/3.4+ and R 3.1+. Java 8 prior to version 8u92 support is deprecated as of...
Read more >Deep Dive into GPU Support in Apache Spark 3.x - Databricks
In this deep dive, we give an overview of accelerator aware task scheduling, columnar data processing support, fractional scheduling, and stage level resource ......
Read more >Support for Spark 3 · Issue #1269 - GitHub
Once the stable Spark 3 is out, we can help you adapt jobserver for Spark 3. There are many previous version upgrade commits...
Read more >Spark Versions Supportability Matrix
Spark versions support different versions of components related to Spark. The following table lists the supported components and versions for the Spark 3...
Read more >Updating Spark 2 applications for Spark 3 | CDP Private Cloud
You must update your Apache Spark 2 applications to run on Spark 3. The Apache Spark documentation provides a migration guide. For instructions...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Yes, closing!
@mrmadira Thanks for the detailed explanation – I will try the above and update the issue. @guykhazma had given me a similar feedback.