Example for schema evolution?
See original GitHub issueHi there,
Looking at the docs, you reference “Compatibility and schema evolution”, but I am unable to find any example on how to achieve schema evolution, which is one of the main benefits of using Avro.
For example: I would like new consumers to be able to read old producer data. Does this library support this?
For example:
V1 schema
{
"type": "record",
"name": "Foo",
"fields": [
{
"name": "f1",
"type": "string"
}
]
}
V2 schema
{
"type": "record",
"name": "Foo",
"fields": [
{
"name": "f1",
"type": "string"
},
{
"name": "f2",
"type": "string",
"default": "",
}
]
}
I would like V2 consumers to have the property f2
with value ""
when reading data produced with schema v1, but this does not happen, since the message is decoded using the old schema
Issue Analytics
- State:
- Created 4 years ago
- Reactions:1
- Comments:5 (2 by maintainers)
Top Results From Across the Web
Schema Evolution and Compatibility - Confluent Documentation
An important aspect of data management is schema evolution. After the initial schema is defined, applications may need to evolve it over time....
Read more >Schema Evolution
Schema evolution is the term used for how the store behaves when Avro schema is changed after data has been written to the...
Read more >Practical Schema Evolution with Avro | by Elliot West - Medium
— Apache Avro has the notion of schema compatibility that allows us to determine whether or not a schema is compatible with one...
Read more >Diving Into Delta Lake: Schema Enforcement & Evolution
Learn how schema enforcement and schema evolution work together on Delta Lake to ensure high quality, reliable data.
Read more >Schema Evolution - Apache Hudi
Schema evolution is a very important aspect of data management. ... Let us walk through an example to demonstrate the schema evolution support...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
I just saw, there is already an open PR for this #137
At our shop we had developed a simpler implementation of schema registry back in the day, since this project was not yet available. Its called Castle (as in "Kafka’s greatest work 😃).
Anyway we’ve recently encountered this issue ourselves, so we did implement reader schemas.
You guys are right, Avsc does support it with
createResolver
, and it’s pretty easy to implement. I’d be happy to give it a go and try to write a PR for it here too, if I had some guidance where you’d think its best to add it.