Allow users to register custom Kryo serializers
See original GitHub issueVersion: release/os/4.4
Problem
We are using persistent immutable maps which are in scope when one of our flows suspend, causing it to be serialized by Kryo. The deserializer fails, causing the following exception to be emitted to the log:
[ERROR] 11:39:49,636 [FiberDeserializationChecker] interceptors.FiberDeserializationChecker. - Encountered unrestorable checkpoint! [errorCode=ewoky4, moreInformationAt=https://errors.corda.net/OS/4.4/ewoky4]
com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
Serialization trace:
dataObject (co.paralleluniverse.fibers.Stack)
stack (net.corda.node.services.statemachine.FlowStateMachineImpl)
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144) ~[kryo-4.0.2.jar:?]
at com.esotericsoftware.kryo.serializers.CompatibleFieldSerializer.read(CompatibleFieldSerializer.java:145) ~[kryo-4.0.2.jar:?]
at com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:782) ~[kryo-4.0.2.jar:?]
at co.paralleluniverse.io.serialization.kryo.ReplaceableObjectKryo.readObjectOrNull(ReplaceableObjectKryo.java:107) ~[quasar-core-0.7.10-jdk8.jar:0.7.10]
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132) ~[kryo-4.0.2.jar:?]
at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:543) ~[kryo-4.0.2.jar:?]
at co.paralleluniverse.fibers.Fiber$FiberSerializer.read(Fiber.java:2137) ~[quasar-core-0.7.10-jdk8.jar:0.7.10]
at co.paralleluniverse.fibers.Fiber$FiberSerializer.read(Fiber.java:2067) ~[quasar-core-0.7.10-jdk8.jar:0.7.10]
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:813) ~[kryo-4.0.2.jar:?]
at co.paralleluniverse.io.serialization.kryo.ReplaceableObjectKryo.readClassAndObject(ReplaceableObjectKryo.java:112) ~[quasar-core-0.7.10-jdk8.jar:0.7.10]
at net.corda.node.serialization.kryo.KryoCheckpointSerializer$deserialize$1$1.invoke(KryoCheckpointSerializer.kt:83) ~[corda-node-4.4.jar:?]
at net.corda.node.serialization.kryo.KryoCheckpointSerializer$deserialize$1$1.invoke(KryoCheckpointSerializer.kt:33) ~[corda-node-4.4.jar:?]
at net.corda.node.serialization.kryo.KryoStreams.kryoInput(KryoStreams.kt:20) ~[corda-node-4.4.jar:?]
at net.corda.node.serialization.kryo.KryoCheckpointSerializer$deserialize$1.invoke(KryoCheckpointSerializer.kt:72) ~[corda-node-4.4.jar:?]
at net.corda.node.serialization.kryo.KryoCheckpointSerializer$deserialize$1.invoke(KryoCheckpointSerializer.kt:33) ~[corda-node-4.4.jar:?]
at net.corda.node.serialization.kryo.KryoCheckpointSerializer$kryo$1.execute(KryoCheckpointSerializer.kt:61) ~[corda-node-4.4.jar:?]
at com.esotericsoftware.kryo.pool.KryoPoolQueueImpl.run(KryoPoolQueueImpl.java:58) ~[kryo-4.0.2.jar:?]
at net.corda.node.serialization.kryo.KryoCheckpointSerializer.kryo(KryoCheckpointSerializer.kt:57) ~[corda-node-4.4.jar:?]
at net.corda.node.serialization.kryo.KryoCheckpointSerializer.deserialize(KryoCheckpointSerializer.kt:71) ~[corda-node-4.4.jar:?]
at net.corda.node.services.statemachine.interceptors.FiberDeserializationChecker$start$2.invoke(FiberDeserializationCheckingInterceptor.kt:101) ~[corda-node-4.4.jar:?]
at net.corda.node.services.statemachine.interceptors.FiberDeserializationChecker$start$2.invoke(FiberDeserializationCheckingInterceptor.kt:51) ~[corda-node-4.4.jar:?]
at kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30) ~[kotlin-stdlib-1.2.71.jar:1.2.71-release-64 (1.2.71)]
Caused by: java.lang.NullPointerException
at com.github.andrewoma.dexx.kollection.MapAdapter.put(ImmutableMap.kt:57) ~[kollection-0.7.jar:?]
at com.github.andrewoma.dexx.kollection.MapAdapter.put(ImmutableMap.kt:45) ~[kollection-0.7.jar:?]
at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:162) ~[kryo-4.0.2.jar:?]
at com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39) ~[kryo-4.0.2.jar:?]
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731) ~[kryo-4.0.2.jar:?]
at co.paralleluniverse.io.serialization.kryo.ReplaceableObjectKryo.readObject(ReplaceableObjectKryo.java:92) ~[quasar-core-0.7.10-jdk8.jar:0.7.10]
at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:391) ~[kryo-4.0.2.jar:?]
at com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.read(DefaultArraySerializers.java:302) ~[kryo-4.0.2.jar:?]
at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:731) ~[kryo-4.0.2.jar:?]
at co.paralleluniverse.io.serialization.kryo.ReplaceableObjectKryo.readObject(ReplaceableObjectKryo.java:92) ~[quasar-core-0.7.10-jdk8.jar:0.7.10]
at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125) ~[kryo-4.0.2.jar:?]
... 21 more
Solution
The solution would be to register a custom serializer with Kryo to override its unsound java.util.Map
serializer. However, I cannot find anything in the Corda documentation on how to do this.
Cause
The cause of the problem is that Kryo’s serializer is generalized in an unsound way to all implementations of java.util.Map
. This doesn’t work for the persistent immutable maps we are using.
Kryo incorrectly assumes that if a class implements java.util.Map
, then it can safely create an instance of it by passing null
for all constructor parameters. In our case, the class is a MapAdapter
which expects a non-null underlying instance of an immutable map implementation as a constructor parameter, and passing null
here breaks the invariants of the class.
Steps to reproduce
- Clone https://github.com/ulrikrasmussen/kryo-unsound-map-deserialization
- Run the integration test. It succeeds, but the exception is shown in the log.
Issue Analytics
- State:
- Created 3 years ago
- Comments:5 (4 by maintainers)
Automatically created Jira issue: CORDA-3717
I think this issue can be closed now: https://docs.r3.com/en/platform/corda/4.6/open-source/cordapp-custom-serializers-checkpoints.html