question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Add support for schema evolution

See original GitHub issue

Does avro4k support reading the default value and putting it into the Avro schema? Found an issue doing an experiment with schema evolution where newer version couldn’t read older version Version “0.1.0”

@Serializable
data class WebhookData(val firstName: String, val lastName: String, val age: Int? = null)
{
  "type" : "record",
  "name" : "WebhookData",
  "namespace" : "com.shoprunner.data.webhook.model",
  "fields" : [ {
    "name" : "firstName",
    "type" : "string"
  }, {
    "name" : "lastName",
    "type" : "string"
  }, {
    "name" : "age",
    "type" : [ "null", "int" ]
  } ]
}

Version “0.2.0”

@Serializable
data class WebhookData(val firstName: String, val lastName: String, val age: Int? = null, val favoriteColor: String? = null)
{
  "type" : "record",
  "name" : "WebhookData",
  "namespace" : "com.shoprunner.data.webhook.model",
  "fields" : [ {
    "name" : "firstName",
    "type" : "string"
  }, {
    "name" : "lastName",
    "type" : "string"
  }, {
    "name" : "age",
    "type" : [ "null", "int" ]
  }, {
    "name" : "favoriteColor",
    "type" : [ "null", "string" ]
  } ]
}

But since default values aren’t set, it can’t read the older data.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:4
  • Comments:15 (8 by maintainers)

github_iconTop GitHub Comments

1reaction
alexhwoodscommented, Feb 28, 2020

Here’s one I’m messing around with:

      test("Is nullable a sufficient default value?") {
         @AvroName("Task")
         @Serializable
         data class Writer(val name: String?)

         @AvroName("Task")
         @Serializable
         data class Reader(@AvroDefault(Avro.NULL) val id: String?, val name: String?)

         val writerSchema = Avro.default.schema(Writer.serializer())
         val readerSchema = Avro.default.schema(Reader.serializer())

         val pairCompatibility = SchemaCompatibility.checkReaderWriterCompatibility(
            readerSchema,
            writerSchema
         )

         pairCompatibility.result shouldBe compatible()
      }
   }

It passes. If you take away the null default value, you get READER_FIELD_MISSING_DEFAULT_VALUE

0reactions
thakecommented, Apr 29, 2020

Closing this issue as it seems to be fixed

Read more comments on GitHub >

github_iconTop Results From Across the Web

Schema Evolution and Compatibility - Confluent Documentation
Schemas evolve in a fully compatible way: old data can be read with the new schema, and new data can also be read...
Read more >
10. Schema Evolution Support - Spring
Spring Cloud Stream provides support for schema evolution so that the data can be evolved over time and still work with older or...
Read more >
Schema Evolution - Apache Hudi
Schema evolution is a very important aspect of data management. Hudi supports common schema evolution scenarios, such as adding a nullable field or...
Read more >
Practical Schema Evolution with Avro | by Elliot West - Medium
Beyond the specification — applying schema evolution to real-world situations ... Asking schema compatibility questions ahead of time can help us avoid ...
Read more >
Adding support for schema evolution | Oracle Coherence 3.5
Adding support for schema evolution ... Now that you know everything you need to know about serialization and POF, let's discuss the closely...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found