[Avro] `JsonMappingException` for union types with multiple Record types
See original GitHub issueI believe this is a different issue from #123 and #164.
In the following example, we have a union schema with 5 record schemas and an abstract class annotated with @Union
as follows:
@Union({
ClassA.class,
ClassB.class,
ClassC.class,
ClassD.class,
ClassE.class
})
abstract class AbstractClass {
public static final Schema SCHEMA = Schema.createUnion(
ClassA.SCHEMA,
ClassB.SCHEMA,
ClassC.SCHEMA,
ClassD.SCHEMA,
ClassE.SCHEMA
);
}
class ClassA extends AbstractClass {}
class ClassB extends AbstractClass {}
class ClassC extends AbstractClass {}
class ClassD extends AbstractClass {}
class ClassE extends AbstractClass {}
Problem 1: serialize as a concrete class and deserialize as an abstract class
final byte[] bytes = avroMapper
.writer(new AvroSchema(ClassA.SCHEMA))
.writeValueAsBytes(new ClassA());
final ClassA result = avroMapper
.readerFor(AbstractClasss.class)
.with(new AvroSchema(AbstractClass.SCHEMA))
.readValue(bytes); // Error
When we serialize a value using the concrete class ClassA
and then try to deserialize it as the AbstractClass
, we get the following error:
com.fasterxml.jackson.core.JsonParseException: Invalid index (36); union only has 5 types
at com.fasterxml.jackson.dataformat.avro.deser.UnionReader._decodeIndex(UnionReader.java:66)
at com.fasterxml.jackson.dataformat.avro.deser.UnionReader.nextToken(UnionReader.java:36)
at com.fasterxml.jackson.dataformat.avro.deser.RootReader.nextToken(RootReader.java:31)
at com.fasterxml.jackson.dataformat.avro.deser.AvroParserImpl.nextToken(AvroParserImpl.java:98)
at com.fasterxml.jackson.databind.ObjectReader._initForReading(ObjectReader.java:355)
at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:1596)
at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1234)
Although serializing and then deserializing the class using the concrete class ClassA
works.
Problem 2: serialize as an abstract class
final byte[] bytes = avroMapper
.writer(new AvroSchema(AbstractClasss.SCHEMA))
.writeValueAsBytes(new ClassA()); // Error
Also, the serialization using the abstract class AbstractClass
does not work:
com.fasterxml.jackson.databind.JsonMappingException: Multiple Record and/or Map types, can not figure out which to use for: [{"x":"y"}]
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._wrapAsIOE(DefaultSerializerProvider.java:509)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:482)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1120)
at com.fasterxml.jackson.databind.ObjectWriter.writeValueAsBytes(ObjectWriter.java:1017)
I believe that both cases are valid and should not cause errors, or maybe I’m missing something.
Issue Analytics
- State:
- Created 4 years ago
- Reactions:1
- Comments:5 (5 by maintainers)
Top Results From Across the Web
Avro Schema: multiple records reference same data type issue
In your JSON data you use the key Customer but you have to use the fully qualified name. So it should be com.example.Customer...
Read more >[#AVRO-2125] AvroRuntimeException: Not a union - ASF JIRA
I have an Avro schema which I am using as part of our Apache Beam pipeline tests, ... JsonMappingException: Not a union: {"type":"record" ......
Read more >com.fasterxml.jackson.dataformat.avro.ser.AvroWriteContext ...
Type.UNION) { schema = _recordOrMapFromUnion(schema); } if (type == Schema. ... new JsonMappingException(null, "Failed to create Record type from "+type, ...
Read more >Registering a union Avro type in Schema Registry
I'm trying to register a union Avro type to Schema Registry, the schema is a union of 2 Avro record types (rec1 and...
Read more >Multiple Event types in same topic with unions - Confluent Forum
I read the article with interest [Multiple Event Types in the Same Kafka Topic ... Invalid schema [ { "name": "io.confluent.examples.avro.
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I did have to do minor tweaking to eliminate now unused method variants that do not take
forValue
(handling ofnull
wrt schema, for the new test, introduced some edge condition… not sure I handled it right). But I assume things work reasonably well at any rate.Thank you, @cowtowncoder!