question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Support for Spark 3.x?

See original GitHub issue

Hi,

Does Stocator work with Spark 3.x? I see there are some dependency conflicts in the versions of Hadoop, etc for Spark 3.x and Stocator. When I create the stocator fat jar and use that with Spark 3.1, I run into Jackson consistency issues, whereas getting a thin jar and using that is missing a lot of dependencies.

Any guidance here will be helpful! Thanks!

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:5 (1 by maintainers)

github_iconTop GitHub Comments

1reaction
raghukiran1224commented, Nov 7, 2021

Yes, closing!

1reaction
raghukiran1224commented, Aug 9, 2021

@mrmadira Thanks for the detailed explanation – I will try the above and update the issue. @guykhazma had given me a similar feedback.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Overview - Spark 3.0.0 Documentation - Apache Spark
Spark runs on Java 8/11, Scala 2.12, Python 2.7+/3.4+ and R 3.1+. Java 8 prior to version 8u92 support is deprecated as of...
Read more >
Deep Dive into GPU Support in Apache Spark 3.x - Databricks
In this deep dive, we give an overview of accelerator aware task scheduling, columnar data processing support, fractional scheduling, and stage level resource ......
Read more >
Support for Spark 3 · Issue #1269 - GitHub
Once the stable Spark 3 is out, we can help you adapt jobserver for Spark 3. There are many previous version upgrade commits...
Read more >
Spark Versions Supportability Matrix
Spark versions support different versions of components related to Spark. The following table lists the supported components and versions for the Spark 3...
Read more >
Updating Spark 2 applications for Spark 3 | CDP Private Cloud
You must update your Apache Spark 2 applications to run on Spark 3. The Apache Spark documentation provides a migration guide. For instructions...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found