KryoException: Unable to find class

I get the exception at the end while trying to upsert the same file more than 12 times ( not sure why 12 ) . The error is given for every partition ( 6 of them - i just pasted the 4th one ) . I am using the same SparkSession to do the upserts. It does not happen if I increase the hoodie.cleaner.commits.retained .
The place where it crashes is on the collect part of this piece of code from HoodieCopyOnWrite:

 List<Tuple2<String, PartitionCleanStat>> partitionCleanStats = jsc
        .parallelize(partitionsToClean, cleanerParallelism)
        .flatMapToPair(getFilesToDeleteFunc(this, config))
        .repartition(cleanerParallelism)                    // repartition to remove skews
        .mapPartitionsToPair(deleteFilesFunc(this)).reduceByKey(
            // merge partition level clean stats below
            (Function2<PartitionCleanStat, PartitionCleanStat, PartitionCleanStat>) (e1, e2) -> e1
                .merge(e2)).collect();

09:26:09.181 [task-result-getter-2] ERROR org.apache.spark.scheduler.TaskResultGetter  - Exception while getting task result
com.esotericsoftware.kryo.KryoException: Unable to find class: hdfs://namenode:8020/test/20190731-091411-373/1564557251826_551/converted/A/4/2c5790b6-eb12-4c15-a84a-f287d9cd9984_1_20190731091435.parquet



A/4
Serialization trace:
deletePathPatterns (com.uber.hoodie.table.HoodieCopyOnWriteTable$PartitionCleanStat)
	at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:160)
	at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:133)
	at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:693)
	at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:804)
	at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
	at com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
	at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
	at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
	at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543)
	at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813)
	at com.twitter.chill.Tuple2Serializer.read(TupleSerializers.scala:42)
	at com.twitter.chill.Tuple2Serializer.read(TupleSerializers.scala:33)
	at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731)
	at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:391)
	at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:302)
	at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813)
	at org.apache.spark.serializer.KryoSerializerInstance.deserialize(KryoSerializer.scala:362)
	at org.apache.spark.scheduler.DirectTaskResult.value(TaskResult.scala:88)
	at org.apache.spark.scheduler.TaskResultGetter$$anon$3$$anonfun$run$1.apply$mcV$sp(TaskResultGetter.scala:72)
	at org.apache.spark.scheduler.TaskResultGetter$$anon$3$$anonfun$run$1.apply(TaskResultGetter.scala:63)
	at org.apache.spark.scheduler.TaskResultGetter$$anon$3$$anonfun$run$1.apply(TaskResultGetter.scala:63)
	at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1991)
	at org.apache.spark.scheduler.TaskResultGetter$$anon$3.run(TaskResultGetter.scala:62)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: hdfs://namenode:8020/test/20190731-091411-373/1564557251826_551/converted/A/4/2c5790b6-eb12-4c15-a84a-f287d9cd9984_1_20190731091435.parquet



A/4
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at com.esotericsoftware.kryo.util.DefaultClassResolver.readName(DefaultClassResolver.java:154)
	... 25 more

[error]    org.apache.spark.SparkException: Job aborted due to stage failure: Exception while getting task result: com.esotericsoftware.kryo.KryoException: Unable to find class: hdfs://namenode:8020/test/20190731-091411-373/1564557251826_551/converted/A/4/2c5790b6-eb12-4c15-a84a-f287d9cd9984_1_20190731091435.parquet



A/4
[error]    Serialization trace:
[error]    deletePathPatterns (com.uber.hoodie.table.HoodieCopyOnWriteTable$PartitionCleanStat) (DAGScheduler.scala:1602)
[error] org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1602)
[error] org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1590)
[error] org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1589)
[error] org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1589)
[error] org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831)
[error] org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831)
[error] org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:831)
[error] org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1823)
[error] org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1772)
[error] org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1761)
[error] org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
[error] org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:642)
[error] org.apache.spark.SparkContext.runJob(SparkContext.scala:2034)
[error] org.apache.spark.SparkContext.runJob(SparkContext.scala:2055)
[error] org.apache.spark.SparkContext.runJob(SparkContext.scala:2074)
[error] org.apache.spark.SparkContext.runJob(SparkContext.scala:2099)
[error] org.apache.spark.rdd.RDD$$anonfun$collect$1.apply(RDD.scala:939)
[error] org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
[error] org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
[error] org.apache.spark.rdd.RDD.withScope(RDD.scala:363)
[error] org.apache.spark.rdd.RDD.collect(RDD.scala:938)
[error] org.apache.spark.api.java.JavaRDDLike$class.collect(JavaRDDLike.scala:361)
[error] org.apache.spark.api.java.AbstractJavaRDDLike.collect(JavaRDDLike.scala:45)
[error] com.uber.hoodie.table.HoodieCopyOnWriteTable.cleanPartitionPaths(HoodieCopyOnWriteTable.java:499)
[error] com.uber.hoodie.table.HoodieCopyOnWriteTable.clean(HoodieCopyOnWriteTable.java:305)
[error] com.uber.hoodie.HoodieWriteClient.clean(HoodieWriteClient.java:1007)
[error] com.uber.hoodie.HoodieWriteClient.commit(HoodieWriteClient.java:546)
[error] com.uber.hoodie.HoodieWriteClient.commit(HoodieWriteClient.java:489)
[error] com.uber.hoodie.HoodieWriteClient.commit(HoodieWriteClient.java:480)
[error] com.uber.hoodie.HoodieSparkSqlWriter$.write(HoodieSparkSqlWriter.scala:155)
[error] com.uber.hoodie.DefaultSource.createRelation(DefaultSource.scala:91)
[error] org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:45)
[error] org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
[error] org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
[error] org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:86)
[error] org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)
[error] org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)
[error] org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
[error] org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
[error] org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
[error] org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)
[error] org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:80)
[error] org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:80)
[error] org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:654)
[error] org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:654)
[error] org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:77)
[error] org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:654)
[error] org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:273)
[error] org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:267)
[error] org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:225)

[arw357]

I tried it and it does not crash - so it’s ok to be cloesd

[arw357]

Yes - nothing changed - I will check again after next release of hoodie and get back to you.