question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

[Avro] `JsonMappingException` for union types with multiple Record types

See original GitHub issue

I believe this is a different issue from #123 and #164.

In the following example, we have a union schema with 5 record schemas and an abstract class annotated with @Union as follows:

@Union({
    ClassA.class,
    ClassB.class,
    ClassC.class,
    ClassD.class,
    ClassE.class
})
abstract class AbstractClass {
    public static final Schema SCHEMA = Schema.createUnion(
        ClassA.SCHEMA,
        ClassB.SCHEMA,
        ClassC.SCHEMA,
        ClassD.SCHEMA,
        ClassE.SCHEMA
    );
}

class ClassA extends AbstractClass {}
class ClassB extends AbstractClass {}
class ClassC extends AbstractClass {}
class ClassD extends AbstractClass {}
class ClassE extends AbstractClass {}

Problem 1: serialize as a concrete class and deserialize as an abstract class

final byte[] bytes = avroMapper
    .writer(new AvroSchema(ClassA.SCHEMA))
    .writeValueAsBytes(new ClassA());

final ClassA result = avroMapper
    .readerFor(AbstractClasss.class)
    .with(new AvroSchema(AbstractClass.SCHEMA))
    .readValue(bytes); // Error

When we serialize a value using the concrete class ClassA and then try to deserialize it as the AbstractClass, we get the following error:

com.fasterxml.jackson.core.JsonParseException: Invalid index (36); union only has 5 types

	at com.fasterxml.jackson.dataformat.avro.deser.UnionReader._decodeIndex(UnionReader.java:66)
	at com.fasterxml.jackson.dataformat.avro.deser.UnionReader.nextToken(UnionReader.java:36)
	at com.fasterxml.jackson.dataformat.avro.deser.RootReader.nextToken(RootReader.java:31)
	at com.fasterxml.jackson.dataformat.avro.deser.AvroParserImpl.nextToken(AvroParserImpl.java:98)
	at com.fasterxml.jackson.databind.ObjectReader._initForReading(ObjectReader.java:355)
	at com.fasterxml.jackson.databind.ObjectReader._bindAndClose(ObjectReader.java:1596)
	at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1234)

Although serializing and then deserializing the class using the concrete class ClassA works.

Problem 2: serialize as an abstract class

final byte[] bytes = avroMapper
    .writer(new AvroSchema(AbstractClasss.SCHEMA))
    .writeValueAsBytes(new ClassA()); // Error

Also, the serialization using the abstract class AbstractClass does not work:

com.fasterxml.jackson.databind.JsonMappingException: Multiple Record and/or Map types, can not figure out which to use for: [{"x":"y"}]

	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._wrapAsIOE(DefaultSerializerProvider.java:509)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:482)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
	at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1396)
	at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1120)
	at com.fasterxml.jackson.databind.ObjectWriter.writeValueAsBytes(ObjectWriter.java:1017)

I believe that both cases are valid and should not cause errors, or maybe I’m missing something.

Issue Analytics

  • State:closed
  • Created 4 years ago
  • Reactions:1
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

1reaction
cowtowncodercommented, Aug 15, 2019

I did have to do minor tweaking to eliminate now unused method variants that do not take forValue (handling of null wrt schema, for the new test, introduced some edge condition… not sure I handled it right). But I assume things work reasonably well at any rate.

0reactions
marcospassoscommented, Aug 15, 2019

Thank you, @cowtowncoder!

Read more comments on GitHub >

github_iconTop Results From Across the Web

Avro Schema: multiple records reference same data type issue
In your JSON data you use the key Customer but you have to use the fully qualified name. So it should be com.example.Customer...
Read more >
[#AVRO-2125] AvroRuntimeException: Not a union - ASF JIRA
I have an Avro schema which I am using as part of our Apache Beam pipeline tests, ... JsonMappingException: Not a union: {"type":"record" ......
Read more >
com.fasterxml.jackson.dataformat.avro.ser.AvroWriteContext ...
Type.UNION) { schema = _recordOrMapFromUnion(schema); } if (type == Schema. ... new JsonMappingException(null, "Failed to create Record type from "+type, ...
Read more >
Registering a union Avro type in Schema Registry
I'm trying to register a union Avro type to Schema Registry, the schema is a union of 2 Avro record types (rec1 and...
Read more >
Multiple Event types in same topic with unions - Confluent Forum
I read the article with interest [Multiple Event Types in the Same Kafka Topic ... Invalid schema [ { "name": "io.confluent.examples.avro.
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found