Avro4s decimal encoding

We need to encode/decode decimals based on Avro docs

Explanation

I’m going to explain the problem through an example. Suppose we have the following definition:

record StockInfo {
  string stockId;
  decimal(10,2) price;
  decimal(5,4) rate;
}

Currently, we do the following:

• Transform this avdl in Scala classes through sbt-avrohugger

case class StockInfo(stockId: String, price: BigDecimal, rate: BigDecimal)

This is serialized/deserialized with the following:

https://github.com/frees-io/freestyle-rpc/blob/876c5f269cf9b8508d9c5c7c79cf597a8dcd8840/modules/internal/src/main/scala/encoders/avro.scala#L68-L80

In the next release, this encoders will be optional (see idlGenMarshallerImports under Plugin Settings) but in any case, this is not Avro compliant (Avro docs)

What we want to do?

We need to find a way for inferring the scale and precision from a Scala type. The encoders and decoders are based on types.

Solutions

Generate wrappers for BigDecimal values in order to generate the custom encoders/decoders.

Approach 1 - Very Ugly

To have thousand of combinations of type BigDecimalS1P1, BigDecimalS1P2, BigDecimalS1P3, …, BigDecimalS2P1, … And they encoders/decoders. Then change sbt-avrohugger for generating these wrappers based on the avdl definition.

case class StockInfo(stockId: String, price: BigDecimalS10P2, rate: BigDecimalS5P4)

Approach 2 - Ugly

Generate the wrappers as part of the code generation in sbt-avrohugger and in a second iteration, generate custom encoders/decoders based on the name in frees-rpc

case class BigDecimalS10P2(bd: BigDecimal)
case class BigDecimalS5P4(bd: BigDecimal)
case class StockInfo(stockId: String, price: BigDecimalS10P2, rate: BigDecimalS5P4)

Approach 3 - Less ugly but still ugly

To have something like this in a avrohugger dependant library:

sealed trait Precision extends Product with Serializable
case object Precision1 extends Precision
case object Precision2 extends Precision
case object Precision3 extends Precision
// ...

sealed trait Scale extends Product with Serializable
case object Scale1 extends Scale
case object Scale2 extends Scale
case object Scale3 extends Scale
// ...

Make sbt-avrohugger to generate the wrapper with these types appended to the BigDecimal types.

case class StockInfo(
  stockId: String, 
  price: BigDecimal with Scale10.type with Precision2.type , 
  rate: BigDecimalwith Scale5.type with Precision4.type)

Then, we could have a custom encoders/decoders for this BigDecimal with extended types. This could be implemented also with shapeless.

The sbt-avrohugger customization would be behind a flag defined as a sbt setting key.

Thoughts?

[pepegar]

Hm… what you want is to define a numeric type with parametrized scale and precision, right?

maybe you could create something like:

sealed trait Nat
object Nat {
  case object Zero extends Nat
  case class Succ(nat: Nat) extends Nat
  
  def toInt(nat: Nat): Int = ...
}

case class Real[S <: Nat, P <: Nat](value: BigDecimal, scale: S, precision: P) 

This would allow you to create something like:

import Nat._
type _One = Succ(Zero)
type _Two = Succ(One)
...

// And later on, create instances as follows:
implicit val FromSchema[Real[_Eleven, _Thirteen]] = ... 

Also, depending on your scala version, I think you can use literal singleton types to do:

case class Real[A, B]...

Real[3, 11](...)

but haven’t tried it.

[juanpedromoreno]

I think it would make sense to move this feature as part of the sbt-avrohugger, to prevent the "reparsing".