`PatchWithOptions` can't specify removal of optional fields

Given a case class with an optional field:

case class Foo(foo: Option[Int])

When deriving a patch for that class:

Decoder[Foo => Foo]

The patch behavior is not correct:

val a = Foo(Some(22))
def patchIt(json: String) = parse(json).flatMap(_.as[Foo]).map(_(a))

patchIt("{}") // Foo(Some(22)) - expected
patchIt("""{"foo": 33}""") // Foo(Some(33)) - expected
patchIt("""{"foo": null}""") // Foo(Some(22)) - should be Foo(None)

The behavior I described above is necessary in order to ever have the ability to remove fields via patching.

It would probably be as simple as defining an explicit decoder for Option[Option[T]] which does what’s described above - if the field isn’t present, it yields None; but if it is present and explicitly null, it yields Some(None). I think that could be considered correct in the general case of Option[Option[T]], not just for patch purposes – though I can’t think of a reason you would use that type outside of the patch use case.

[jeremyrsmith]

@Tvaroh what you’re seeing isn’t related to this issue. The null values in JSON representation arise from a property of the Printer, which is configurable. The default printers have dropNullKeys set to false, but you can create your own printer with dropNullKeys set to true, and use that to output the JSON:

//before
myJson.noSpaces  //includes nulls

//after
myJson.pretty(Printer.noSpaces.copy(dropNullKeys = true)) // nulls are not included

[bergmark]

Thanks @jeremyrsmith, this is useful to me.

But note that using a custom printer makes the assumption that you want to treat all null keys equally. This is something I think should be decided on the instance level instead.

e.g. given C(a = X(None), b = Y(None)) we by default get{ "a" : { "x" : null }, "b" : { "y": null } }. It might be necessary to turn this into { "a" : {}, "b" : { y : null } } and this decision should IMO be made in the instance declarations of X and Y. This is possible using aeson.