Serialization macros intermittently fail

I have a relatively simple case class graph to serialize, with one instance of ‘sealed trait’ implemented by multiple case classes.

It fails when compiled with my application, as described below. However, when I remove all other code and only compile the case classes and Json.scala, everything works fine!

The thing needed to make it work is to remove the code (Scala Android app) not related to upickle in any way!

When I compile the whole code through sbt, I get very many messages such as:

The referenced trait [[Answer]] does not have any sub-classes. This may happen due to a limitation of scalac (SI-7046) given that the trait is not in the same package. If this is the case, the hierarchy may be defined using integer constants.

and then a few compile messages such as

[error] C:\projects\myproject\android\src\main\scala\package\SynchronizationActivity.scala:28: exception during macro expansion:
[error] java.lang.AssertionError: assertion failed
[error]         at scala.Predef$.assert(Predef.scala:151)
[error]         at upickle.Macros$.macroWImpl(Macros.scala:49)
[error]         at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
[error]         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error]         at java.lang.reflect.Method.invoke(Method.java:606)
[error]         at scala.reflect.macros.runtime.JavaReflectionRuntimes$JavaReflectionResolvers$$anonfun$resolveJavaReflectionRuntime$2.apply(JavaReflectionRuntimes.scala:34)
[error]         at scala.reflect.macros.runtime.JavaReflectionRuntimes$JavaReflectionResolvers$$anonfun$resolveJavaReflectionRuntime$2.apply(JavaReflectionRuntimes.scala:22)
[error]         at scala.tools.nsc.typechecker.Macros$class.macroExpandWithRuntime(Macros.scala:755)
[error]     info(s"This would be sent to server: ${Serialize(qr)}")
[error]                                                     ^
[error] 15 errors found
[error] (compile:compile) Compilation failed

The trait it has a problem with is roughly

sealed trait Answer 
case class AnswerWithDefects(defects: Option[Seq[Defect]] = None) extends Answer
case class AnswerYesNo(expectedAnswer: Boolean, answer: Option[Boolean] = None) extends Answer 

The code doing the serialization is

package mypackage.json
import upickle._

object Serialize {
  def apply[T : Writer](value: T) = write(value)
}

object Deserialize {
  def apply[T : Reader](json: String) = read[T](json)
}

I also tried adding the following in there after the upickle import, but the results are the same:

import mypackage.{AnswerYesNo, AnswerWithDefects, Answer}
object JsonImplicits {
  val classNameKey = "className"
  val dataKey = "data"

  implicit val answerWriter = Writer[Answer] {
        case answer =>
          val data = answer match {
            case x: AnswerWithDefects => writeJs[AnswerWithDefects](x)
            case x: AnswerYesNo => writeJs[AnswerYesNo](x)
          }

          Js.Obj(
            classNameKey -> Js.Str(answer.getClass.getName),
            dataKey -> data
          )
      }

      implicit val answerRead = Reader[Answer] {
        case o @ Js.Obj(_) =>
          o(classNameKey) match {
            case Js.Str(className) =>
              val dataJson = o(dataKey)

              val data = if (className == classOf[AnswerWithDefects].getName) {
                readJs[AnswerWithDefects](dataJson)
              } else if (className == classOf[AnswerYesNo].getName) {
                readJs[AnswerYesNo](dataJson)
              } else {
                throw new RuntimeException(s"Did not recognize $className")
              }

              data
          }
      }
}

import JsonImplicits._

I appreciate the root cause is the scalac defect SI-7046, and that it’s not getting fixed any time soon, but do you have any workaround ideas to make it work?

I had hoped that me providing the implicits would make it not resort to reflection when analyzing the Answer class, but it still fails.

[tindzk]

Sure. The tree is specified as follows (Prop is a sealed trait hierarchy):

sealed trait Tree
object Tree {
  case class Node(id: Int, prop: Prop, children: Seq[Tree]) extends Tree
  case class Leaf(id: Int, prop: Prop, text: String) extends Tree
}

Before I figured out the above workaround I was using:

object Picklers {
  import upickle.Aliases._
  import upickle.Js

  implicit def TreeR: R[Tree] = R[Tree](
    NodeR.read.orElse(LeafR.read)
  )

  implicit def TreeW: W[Tree] = W[Tree] {
    case x @ Tree.Node(_, _, _) => NodeW.write(x)
    case x @ Tree.Leaf(_, _, _) => LeafW.write(x)
  }

  implicit def NodeR: R[Tree.Node] = R[Tree.Node] {
    case Js.Arr(Js.Num(0), a, b, c) => Tree.Node(
      upickle.readJs[Int](a),
      upickle.readJs[Prop](b),
      upickle.readJs[Seq[Tree]](c)
    )
  }

  implicit def LeafR: R[Tree.Leaf] = R[Tree.Leaf] {
    case Js.Arr(Js.Num(1), a, b, c) => Tree.Leaf(
      upickle.readJs[Int](a),
      upickle.readJs[Prop](b),
      upickle.readJs[String](c)
    )
  }

  implicit def NodeW: W[Tree.Node] = W[Tree.Node] { x: Tree.Node =>
    Js.Arr(
      upickle.writeJs(0),
      upickle.writeJs(x.id),
      upickle.writeJs(x.prop),
      upickle.writeJs(x.children)
    )
  }

  implicit def LeafW: W[Tree.Leaf] = W[Tree.Leaf] { x: Tree.Leaf =>
    Js.Arr(
      upickle.writeJs(1),
      upickle.writeJs(x.id),
      upickle.writeJs(x.prop),
      upickle.writeJs(x.text)
    )
  }
}

Edit: After discussing the matter again with Kyle, we found that the macros from the previous posting may even fail when the manual reader/writer is at the declaration site.

If this is the case, the only possible solution will be to mimic the behaviour of the macro so that knownDirectSubclasses isn’t queried in the first place:

sealed trait Base
object Base {
  case class Child1() extends Base
  case class Child2() extends Base
  case class Child3() extends Base

  import upickle._

  implicit val BaseR: Reader[Base] = Reader[Base](
    implicitly[Reader[Child1]].read
      .orElse(implicitly[Reader[Child2]].read)
      .orElse(implicitly[Reader[Child3]].read)
  )

  implicit val BaseW: Writer[Base] = Writer[Base] {
    case x: Child1 => writeJs(x)
    case x: Child2 => writeJs(x)
    case x: Child3 => writeJs(x)
  }
}

[lihaoyi]

I’m gonna close this in preference for https://github.com/lihaoyi/upickle/issues/31