Play json formats which serialize traits do not work with Lagom's persistence layer

Lagom Version: 1.3.x

API: Scala

Problem:

The process of selecting a potential serializer differs between Akka and Lagom. As a result instances of Format[T] which serve as serializers for traits/base classes are either not used or not found when registered with a JsonSerializerRegistry (for use in Lagom’s persistence layer).

Details:

Given a trait and case class like this:

sealed trait UserEvent

case class UserUpdatedEvt(email: String,
                          passwordHash: String,
                          firstName: String,
                          lastName: String,
                          timestamp: Instant) extends UserEvent

// and more case classes which extend UserEvent...

and a Format[T] as well as a JsonSerializationRegistry:

object UserEvent {
  implicit val format: Format[UserEvent] = new Format[UserEvent] {
    override def reads(json: JsValue): JsResult[UserEvent] = {
      // determine which case class we intend to deserialize by 
     // inspecting the json and return JsSuccess
      ???
    }

    override def writes(o: UserEvent): JsValue = {
      // use a match statement to determine which case class we are
      // serializing and add an extra field (e.g. 'type') to the resulting 
      // JsObject (for use in identifying the correct type during deserialization)
      ???
    }
  }
}

object UserSerializerRegistry extends JsonSerializerRegistry {
  override def serializers: Seq[JsonSerializer[_]] = Seq(
    JsonSerializer[UserEvent]
  )
}

When attempting to persist an instance of UserUpdatedEvt

If the akka.actor.serialization.bindings.java.io.Serializable configuration setting (in application.conf) is left to its default setting Akka will:

• find both a java serializer as well as the provided PlayJsonSerializer
• produce the warnings below and use the java serializer:

[warn] a.s.Serialization(akka://account-service-impl-application) - Multiple serializers found for class com.account.impl.UserUpdatedEvt, choosing first: Vector((interface java.io.Serializable,akka.serialization.JavaSerializer@36a7e601), (interface com.account.impl.UserEvent,com.lightbend.lagom.scaladsl.playjson.PlayJsonSerializer@73a57eb1
[warn] a.s.Serialization(akka://account-service-impl-application) - Using the default Java serializer for class [com.account.impl.UserUpdatedEvt] which is not recommended because of performance implications. Use another serializer or disable this warning using the setting 'akka.actor.warn-about-java-serializer-usage'

Note that even though it defaults to java serialization it does actually find the PlayJsonSerializer for the UserEvent trait. This is because Akka finds potential serializers in its registry by calling isAssignableFrom as seen here: Serialization.scala#L233

---

If the akka.actor.serialization.bindings.java.io.Serializable configuration setting (in application.conf) is set to none Akka will:

• find the PlayJsonSerializer and call toBinary as seen here: Serialization.scala#L114
• PlayJsonSerializer will then attempt to use the manifestClassName to look up the corresponding serializer in a Map[String, Format[AnyRef]] as seen here: PlayJsonSerializer.scala#L48
• The look up will fail because the manifestClassName is "UserUpdatedEvt" not "UserEvent"
• An exception is thrown producing an error like:

java.lang.RuntimeException: Missing play-json serializer for [com.account.impl.UserUpdatedEvt]

---

Potential fixes:

For the java serialization default selection issue:

• After fixing the lookup issue (see below) set akka.actor.serialization.bindings.java.io.Serializable to none if a non-empty PlayJsonSerializer is specified in the LagomApplication (all or nothing approach)
• After fixing the lookup issue (see below) add some documentation stating that if trait/base class Format[T]s will be used in the persistence layer then akka.actor.serialization.bindings.java.io.Serializable should be manually set to none in appliaction.conf
• Find someway to get PlayJsonSerializer to have higher priority (e.g. Jsonable 💩)

For the PlayJsonSerializer serializer lookup issue

• Implement a similar selection process using isAssignableFrom in PlayJsonSerializer. The downside here is that we’d be repeating the work already done by Akka.
• To avoid the repeated reflection of solution 1 we could instead
  • add a def toBinary(o: AnyRef, clazz: Class[_]): Array[Byte] method to the Serializer trait in Akka and have Akka call that instead - passing along the Class[_] which lead to the selection of the serializer being called (in the example above this would be the UserEvent trait's Class[_]`).
  • Override this new method in PlayJsonSerializer and use the provided Class[_] during Format[T] lookup.
  • For backwards compatibility the default implementation of this new method should be:

def toBinary(o: AnyRef, clazz: Class[_]): Array[Byte] = toBinary(o)

I have yet to look at the deserialization side of this problem so I’m not sure if there will be similar issues to resolve. I’m looking to start a discussion around this to see if anyone has ideas for a better solution.

Final Note: I ran into this problem while using play-json-derived-codecs for PersistentEntity serialization/deserialization (in an attempt to reduce boiler plate). I’m aware that this problem can be avoided by ditching the derived codecs and declaring a unique Format[T] for each case class. However after figuring out the root cause I’m inclined to say that Akka is more-or-less correctly assuming that Serializer.toBinary should be able to handle covariant arguments. After all this is the same assumption we make when calling any function that takes some object interface as an argument. The rules of polymorphism are just a little convoluted here by the fact that we first register the serializer’s “actual argument type” with Akka - and then later implement an interface which takes anAnyRef.

[octonato]

Thanks @crfeliz. The sample will be very helpful. I will have a look on it tomorrow.

[dinhnguyen92]

We have the same problem with Lagom 1.6. Our Lagom services are deployed in Docker containers in AWS ECS, and use Cassandra in AWS Keyspaces. The only workaround that works for us, as suggested by @blanchet4forte , is to bypass SerializerRegistry altogether and instead use Akka jackson-json serializers exclusively. This does require us to convert events and commands that are case objects into case classes.