mmlspark.lightgbm._LightGBMClassifier does not exist
See original GitHub issueDescribe the bug mmlspark.lightgbm._LightGBMClassifier does not exist
To Reproduce
I git cloned the repo and sys.path.append the mmlspark python path, import mmlspark
has no issue, but the classifier inside can’t be used
There is no clear instruction on how to install mmlspark for python.
spark = pyspark.sql.SparkSession.builder.appName("MyApp") \
.config("spark.jars.packages", "com.microsoft.ml.spark:mmlspark_2.11:1.0.0-rc1") \
.getOrCreate()
from mmlspark.lightgbm import LightGBMClassifier
model = LightGBMClassifier(learningRate=0.3,
numIterations=100,
numLeaves=31).fit(train)
spark.stop()
Expected behavior from mmlspark.lightgbm import LightGBMClassifier should work
Info (please complete the following information):
- MMLSpark Version: from latest repo
- Spark Version 2.4.4
- Spark Platform: custom platform not on azure
** Stacktrace**
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-9-1b16cbc5ea7e> in <module>
5 .config("spark.jars.packages", "com.microsoft.ml.spark:mmlspark_2.11:1.0.0-rc1") \
6 .getOrCreate()
----> 7 from mmlspark.lightgbm import LightGBMClassifier
8 model = LightGBMClassifier(learningRate=0.3,
9 numIterations=100,
/mnt/user-home/git/mmlspark/src/main/python/mmlspark/lightgbm/LightGBMClassifier.py in <module>
9 basestring = str
10
---> 11 from mmlspark.lightgbm._LightGBMClassifier import _LightGBMClassifier
12 from mmlspark.lightgbm._LightGBMClassifier import _LightGBMClassificationModel
13 from pyspark import SparkContext
ModuleNotFoundError: No module named 'mmlspark.lightgbm._LightGBMClassifier'
Additional context I tried this on Jupyter on a Linux machine. Does this only work on Azure?
Issue Analytics
- State:
- Created 4 years ago
- Reactions:4
- Comments:34 (8 by maintainers)
Top Results From Across the Web
mmlspark.lightgbm._LightGBMClassifier does not exist #718
Describe the bug mmlspark.lightgbm._LightGBMClassifier does not exist To Reproduce I git cloned the repo and sys.path.append the mmlspark python path, ...
Read more >mmlspark.lightgbm package — Microsoft Machine Learning ...
Load the model from a native LightGBM model string. Save the booster as string format to a local or WASB remote location.
Read more >how do I install parckage(such as mmlspark) to CDH cluster ...
Here, I think this error is because mmlspark python port can not load the jar , which causes Py4JJavaError. But I have no...
Read more >Microsoft/MMLSpark - Gitter
Hey folks, is there anyone tried to build mmlspark on your own desktop ? some unit tests could not pass for the LightGBM...
Read more >LightGBM - Overview | SynapseML - Microsoft Open Source
Cross platform LightGBM on Spark is available on Spark, PySpark, and SparklyR ... not, we could build a binary classification model with LightGBMClassifier....
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
So is there any solutions for this problem?
Getting the same error as others on this thread. I was going through the
LightGBM
example here: