question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Running EventHubsStructuredStreamingExample in cluster: cannot find data source "eventhubs"

See original GitHub issue

When submitting a simple job with just the sample code included, I seem to always get the following error on runtime when val inputStream = sparkSession.readStream.format("eventhubs").options(eventhubParameters).load() executes:

java.lang.ClassNotFoundException: Failed to find data source: eventhubs. Please find packages at http://spark.apache.org/third-party-projects.html

Any ideas on how to get the sample code EventHubsStructuredStreamingExample working?

Issue Analytics

  • State:closed
  • Created 6 years ago
  • Comments:5 (2 by maintainers)

github_iconTop GitHub Comments

2reactions
p2bauercommented, Sep 12, 2017

@sabeegrewal that was absolutely it, works now. thanks very much for your help. 👍

0reactions
sabeegrewalcommented, Sep 12, 2017

It needs to be like this:

<dependency>
    <groupId>com.microsoft.azure</groupId>
    <artifactId>spark-streaming-eventhubs_2.11</artifactId>
    <version>2.1.2</version>
</dependency>

So, instead of spark-streaming-eventhubs_connector_2.11, you wanna do spark-streaming-eventhubs_2.11. Let me know if that works!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Spark job with eventhubs in Databricks ends: Failed to find ...
Bug Report: When running the following spark job in Databricks: val df ... ClassNotFoundException: Failed to find data source: eventhubs.
Read more >
Troubleshoot connectivity issues - Azure Event Hubs
This article provides information on troubleshooting connectivity issues with Azure Event Hubs.
Read more >
Could not instantiate EventHubSourceProvider for Azure ...
Install com.microsoft.azure:azure-eventhubs-spark_2.11:2.3.15 library version from Maven Central; Restart cluster; Validate by re-running code ...
Read more >
Azure Event Hubs data source discovery
When running a discovery against an Azure Event Hub namespace defined within your Data source configuration, you can specify query parameters to ...
Read more >
Azure Event Hubs | Databricks on AWS
Learn how to use Azure Event Hubs as a source and sink for streaming data in Databricks.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found