Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Python API Spark 1.5.2 Magellen 1.0.4 Snapshot on Databricks not working

See original GitHub issue

Hi there! I compiled Magellen from this branch https://github.com/harsha2010/magellan/pull/46 (halfabrane:MAGELLAN-SPARK1.6 / 1.0.4 SNAPSHOT).

I am using Databricks and when I try to import modules form magellen. Scala version works fine.

ImportError: No module named magellan.types
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-b8ee42532bbb> in <module>()
----> 1 from magellan.types import Point, Polygon
      2 from pyspark.sql import Row, SQLContext
      3 import pyspark.sql.types
      4 import pyspark.sql.functions

ImportError: No module named magellan.types

I have uploaded the dependencies

Issue Analytics

State:
Created 7 years ago
Comments:31 (9 by maintainers)

Top GitHub Comments

3reactions

harsha2010commented, Mar 16, 2018

@tomplex , thanks! I’m working on the Python API for the next release. #210 Spark 2.3 requires a few changes in Magellan so might cut a separate release for Spark 2.3+ and Spark 2.2 and prior . Will keep you posted!

1reaction

Mahshidscommented, Sep 29, 2017

Hi @harsha2010. Does Magellan work with Python and Spark 2.1?

Top Results From Across the Web

Python API for Magellan · Issue #210 - GitHub

The Python API was broken after 1.0.4 release (incompatibility with ... Python API Spark 1.5.2 Magellen 1.0.4 Snapshot on Databricks not ...

Call the Databricks REST API with Python

Learn how to use Python to call the Databricks REST API through the Databricks CLI source code.

Spark image failed to download or does not exist

Cause Unexpected state for cluster: INVALID_ARGUMENT(CLIENT_ERROR): databricks_error_message: Container setup failed because of an invalid request: Spark image ...

Is there any way to save the notebook in the "Results Only ...

Hi! I'm looking for a solution to save a notebook in HTML format that has the "Results Only" view (without the executed code)....

Simple, Reliable Upserts and Deletes on Delta Lake Tables ...

DeltaSparkSessionExtension to enable Delta Lake SQL commands within Apache Spark; this is not required for Python or Scala API calls.