question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Accumulating nested decoding failures

See original GitHub issue

Here’s an example where the obvious way to write a Decoder causes error accumulation to disappear:

ealed trait T

@JsonCodec
case class A(a1: Int, a2: Int) extends T

@JsonCodec
case class B(b1: Int, b2: Int) extends T

object Main extends App {

  val normalDecoder: Decoder[T] =
    Decoder.instance(cursor => {
      cursor.downField("type").as[String].flatMap {
        case "a" => cursor.as[A]
        case "b" => cursor.as[B]
        case other: String => Left(DecodingFailure(s"Unknown type: '$other'", Nil))
      }
    })

  println(decodeAccumulating[T]("""{ "type": "a" }""")(normalDecoder))
}

=> Invalid(NonEmptyList(DecodingFailure(Attempt to decode value on failed cursor, List(DownField(a1)))))

Decoder.instance does not allow you to have several error messages so we only get the first error.

We tried to fix this in our application and AFAICT the only way to do so is to create a custom Decoder, something that is only possible in the io.circe namespace as Decoder.decodeAccumulating is private[circe]. This is what I ended up with:

package io.circe

import cats.data.{Validated, ValidatedNel}
import cats.data.Validated.Invalid
import io.circe.Decoder.Result
import io.circe.parser.decodeAccumulating
import io.circe.generic.JsonCodec

sealed trait T

@JsonCodec
case class A(a1: Int, a2: Int) extends T

@JsonCodec
case class B(b1: Int, b2: Int) extends T

object AccMain extends App {

  def make[Ty](r: HCursor => AccumulatingDecoder.Result[Ty]): Decoder[Ty] =
    new Decoder[Ty] {
      override private[circe] def decodeAccumulating(c: HCursor): AccumulatingDecoder.Result[Ty] =
        r(c)

      override def apply(c: HCursor): Result[Ty] =
        r(c).toEither.left.map(_.head) // Should be optimizable.
    }

  val accDecoder: Decoder[T] =
    make[T] { (cursor: HCursor) =>
      val y: ValidatedNel[DecodingFailure, String] = Validated.fromEither(cursor.downField("type").as[String]).toValidatedNel
      y.andThen {
        case "a" => Decoder[A].accumulating(cursor)
        case "b" => Decoder[B].accumulating(cursor)
        case other: String => Invalid(DecodingFailure(s"Unknown type: '$other'", Nil)).toValidatedNel
      }
    }

  println(decodeAccumulating[T]("""{ "type": "a" }""")(accDecoder))
}

object OriginalMain extends App {

  val normalDecoder: Decoder[T] =
    Decoder.instance(cursor => {
      cursor.downField("type").as[String].flatMap {
        case "a" => cursor.as[A]
        case "b" => cursor.as[B]
        case other: String => Left(DecodingFailure(s"Unknown type: '$other'", Nil))
      }
    })

  println(decodeAccumulating[T]("""{ "type": "a" }""")(normalDecoder))
}

AccMain => Invalid(NonEmptyList(DecodingFailure(Attempt to decode value on failed cursor, List(DownField(a1))), DecodingFailure(Attempt to decode value on failed cursor, List(DownField(a2)))))

I have a couple of questions:

  • Is there currently a better way to do this when writing a manual instance?
  • Should a function like make be included in circe?

Issue Analytics

  • State:open
  • Created 6 years ago
  • Comments:10 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
gpoiriercommented, Aug 24, 2018

So if I understand correctly, even using accumulating decoder, it’s not possible to have a single decoder return two failures directly?

0reactions
abhijitSingh86commented, Aug 27, 2020

+1

Read more comments on GitHub >

github_iconTop Results From Across the Web

Decoding structured JSON arrays with circe in Scala
If the input isn't an array at all, or is an empty array, this operation will fail, and we'll get a useful error...
Read more >
Flattening a nested JSON response into a single struct with ...
In this post I'll show you how you can use nested containers to decode nested JSON data into a flat struct with a...
Read more >
Fig. S6. The two basis elements for ‗accumulation of failures ...
(A) The basis element that accumulates failures and persists with rewards (grey) is similar to the consecutive failures (pink) for all sequences where...
Read more >
Module dj - HexDocs
Note that those atoms must already exist, or decoding will fail. ... at/2, Instruct a decode to match a value in a nested...
Read more >
Protocol Buffer Basics: C++ - Google Developers
If libprotobuf is compiled in debug mode, serializing an uninitialized message will cause an assertion failure. In optimized builds, the check is skipped...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found