Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
key not found: org.apache.spark.ml.feature.ImputerModel
See original GitHub issueIssue Description
According to the doc, imputer is supported. But I get this error when trying to save bundlefile. Here are my dependency versions: ‘‘spark 2.2.0’’ “ml.combust.mleap” %% “mleap-runtime” % “0.9.5”, “ml.combust.mleap” %% “mleap-spark” % “0.9.5” I don’t know what to do, can u help me, please~
Exception in thread "main" java.util.NoSuchElementException: key not found: org.apache.spark.ml.feature.ImputerModel
at scala.collection.MapLike$class.default(MapLike.scala:228)
at scala.collection.AbstractMap.default(Map.scala:59)
at scala.collection.MapLike$class.apply(MapLike.scala:141)
at scala.collection.AbstractMap.apply(Map.scala:59)
at ml.combust.bundle.BundleRegistry.opForObj(BundleRegistry.scala:84)
at ml.combust.bundle.serializer.GraphSerializer$$anonfun$writeNode$1.apply(GraphSerializer.scala:31)
at ml.combust.bundle.serializer.GraphSerializer$$anonfun$writeNode$1.apply(GraphSerializer.scala:30)
at scala.util.Try$.apply(Try.scala:192)
at ml.combust.bundle.serializer.GraphSerializer.writeNode(GraphSerializer.scala:30)
at ml.combust.bundle.serializer.GraphSerializer$$anonfun$write$2.apply(GraphSerializer.scala:21)
at ml.combust.bundle.serializer.GraphSerializer$$anonfun$write$2.apply(GraphSerializer.scala:21)
at scala.collection.IndexedSeqOptimized$class.foldl(IndexedSeqOptimized.scala:57)
at scala.collection.IndexedSeqOptimized$class.foldLeft(IndexedSeqOptimized.scala:66)
at scala.collection.mutable.WrappedArray.foldLeft(WrappedArray.scala:35)
at ml.combust.bundle.serializer.GraphSerializer.write(GraphSerializer.scala:20)
at org.apache.spark.ml.bundle.ops.PipelineOp$$anon$1.store(PipelineOp.scala:21)
at org.apache.spark.ml.bundle.ops.PipelineOp$$anon$1.store(PipelineOp.scala:14)
at ml.combust.bundle.serializer.ModelSerializer$$anonfun$write$1.apply(ModelSerializer.scala:87)
at ml.combust.bundle.serializer.ModelSerializer$$anonfun$write$1.apply(ModelSerializer.scala:83)
at scala.util.Try$.apply(Try.scala:192)
at ml.combust.bundle.serializer.ModelSerializer.write(ModelSerializer.scala:83)
at ml.combust.bundle.serializer.NodeSerializer$$anonfun$write$1.apply(NodeSerializer.scala:85)
at ml.combust.bundle.serializer.NodeSerializer$$anonfun$write$1.apply(NodeSerializer.scala:81)
at scala.util.Try$.apply(Try.scala:192)
at ml.combust.bundle.serializer.NodeSerializer.write(NodeSerializer.scala:81)
at ml.combust.bundle.serializer.BundleSerializer$$anonfun$write$1.apply(BundleSerializer.scala:34)
at ml.combust.bundle.serializer.BundleSerializer$$anonfun$write$1.apply(BundleSerializer.scala:29)
at scala.util.Try$.apply(Try.scala:192)
at ml.combust.bundle.serializer.BundleSerializer.write(BundleSerializer.scala:29)
at ml.combust.bundle.BundleWriter.save(BundleWriter.scala:26)
at com.zhihu.saturn.offline.process.CalculateLRModel$$anonfun$training$2.apply(CalculateLRModel.scala:161)
at com.zhihu.saturn.offline.process.CalculateLRModel$$anonfun$training$2.apply(CalculateLRModel.scala:160)
at resource.AbstractManagedResource$$anonfun$5.apply(AbstractManagedResource.scala:88)
at scala.util.control.Exception$Catch$$anonfun$either$1.apply(Exception.scala:125)
at scala.util.control.Exception$Catch$$anonfun$either$1.apply(Exception.scala:125)
at scala.util.control.Exception$Catch.apply(Exception.scala:103)
at scala.util.control.Exception$Catch.either(Exception.scala:125)
at resource.AbstractManagedResource.acquireFor(AbstractManagedResource.scala:88)
at resource.ManagedResourceOperations$class.apply(ManagedResourceOperations.scala:26)
at resource.AbstractManagedResource.apply(AbstractManagedResource.scala:50)
at resource.ManagedResourceOperations$class.acquireAndGet(ManagedResourceOperations.scala:25)
at resource.AbstractManagedResource.acquireAndGet(AbstractManagedResource.scala:50)
at resource.ManagedResourceOperations$class.foreach(ManagedResourceOperations.scala:53)
at resource.AbstractManagedResource.foreach(AbstractManagedResource.scala:50)
at com.zhihu.saturn.offline.process.CalculateLRModel$.training(CalculateLRModel.scala:160)
at com.zhihu.saturn.offline.process.CalculateLRModel$.run(CalculateLRModel.scala:51)
at com.zhihu.saturn.offline.process.CalculateLRModel$.main(CalculateLRModel.scala:225)
at com.zhihu.saturn.offline.process.CalculateLRModel.main(CalculateLRModel.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:755)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Issue Analytics
- State:
- Created 5 years ago
- Comments:9 (2 by maintainers)
Top Results From Across the Web
mleap support Spark ML Imputer - serialization - Stack Overflow
util.NoSuchElementException: key not found: org.apache.spark.ml.feature.ImputerModel . Does this mean that the Imputer is not supported?
Read more >Extracting, transforming and selecting features - Apache Spark
Extracting, transforming and selecting features. This section covers algorithms for working with features, roughly divided into these groups:.
Read more >org.apache.spark.ml.feature.ImputerModel
A list of advanced, expert-only (hyper-)parameter keys this algorithm can take. Users can set and get the parameter values through setters and getters, ......
Read more >Spark 3.3.1 ScalaDoc - org.apache.spark.ml.feature.ImputerModel
Spark 3.3.1 ScalaDoc - org.apache.spark.ml.feature.ImputerModel.
Read more >Extracting, transforming and selecting features - Spark 2.3.1 ...
HashingTF; import org.apache.spark.ml.feature. ... When an a-priori dictionary is not available, CountVectorizer can be used as an Estimator to extract the ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I have the same issue working with Pyspark 2.3.0. mleap-spark-extension are not available in python from what I saw.
@caesarjuly Try adding “ml.combust.mleap” %% “mleap-spark-extension” % “0.9.5” as a dependency and use
from there instead.
I believe that the out of the box Spark transformer can work on multiple columns and that isn’t supported at the moment in MLeap. The transformer from mleap-spark-extension works the same as Spark’s, with the additional restriction that it works on just a single column.