question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Union types that contain a type that require an avrocschema.types specifier fail to generate schema

See original GitHub issue

Describe the bug A field with a Union type, where the Union contains a type that requires a dataclasses_avroschema.types default to provide additional metadata (Decimal, Enum, etc) fails to generate a schema.

To Reproduce

@dataclass
class TestEnum(AvroModel):
    OptionalEnum: typing.Optional[types.Enum] = types.Enum(['Mr', 'Mrs', 'Miss', 'Ms'])
    # DateEnum = typing.Union[datetime, types.Enum] = types.Enum(['Mr', 'Mrs', 'Miss', 'Ms'])
    # DecimalEnum = typing.Union[decimal.Decimal, types.Enum] = (types.Decimal(precision=2, scale=1), types.Enum(['Mr', 'Mrs', 'Miss', 'Ms']))

print(TestEnum.avro_schema())

Generates the following stacktrace

faust_1  | Traceback (most recent call last):
faust_1  |   File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
faust_1  |     return _run_code(code, main_globals, None,
faust_1  |   File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
faust_1  |     exec(code, run_globals)
faust_1  |   File "/app/proj/__main__.py", line 17, in <module>
faust_1  |     print(TestEnum.avro_schema())
faust_1  |   File "/usr/local/lib/python3.8/site-packages/dataclasses_avroschema/schema_generator.py", line 58, in avro_schema
faust_1  |     return json.dumps(cls.generate_schema(schema_type=AVRO).render())
faust_1  |   File "/usr/local/lib/python3.8/site-packages/dataclasses_avroschema/schema_generator.py", line 46, in generate_schema
faust_1  |     cls.schema_def = cls._generate_avro_schema()
faust_1  |   File "/usr/local/lib/python3.8/site-packages/dataclasses_avroschema/schema_generator.py", line 54, in _generate_avro_schema
faust_1  |     return schema_definition.AvroSchemaDefinition("record", cls.klass, metadata=cls.metadata)
faust_1  |   File "<string>", line 9, in __init__
faust_1  |   File "/usr/local/lib/python3.8/site-packages/dataclasses_avroschema/schema_definition.py", line 55, in __post_init__
faust_1  |     self.fields = self.parse_dataclasses_fields()
faust_1  |   File "/usr/local/lib/python3.8/site-packages/dataclasses_avroschema/schema_definition.py", line 60, in parse_dataclasses_fields
faust_1  |     return self.parse_fields()
faust_1  |   File "/usr/local/lib/python3.8/site-packages/dataclasses_avroschema/schema_definition.py", line 63, in parse_fields
faust_1  |     return [
faust_1  |   File "/usr/local/lib/python3.8/site-packages/dataclasses_avroschema/schema_definition.py", line 64, in <listcomp>
faust_1  |     AvroField(
faust_1  |   File "/usr/local/lib/python3.8/site-packages/dataclasses_avroschema/fields.py", line 820, in field_factory
faust_1  |     return container_klass(  # type: ignore
faust_1  |   File "<string>", line 10, in __init__
faust_1  |   File "/usr/local/lib/python3.8/site-packages/dataclasses_avroschema/fields.py", line 355, in __post_init__
faust_1  |     self.unions = self.generate_unions_type()
faust_1  |   File "/usr/local/lib/python3.8/site-packages/dataclasses_avroschema/fields.py", line 382, in generate_unions_type
faust_1  |     unions.append(default_field.get_avro_type())
faust_1  |   File "/usr/local/lib/python3.8/site-packages/dataclasses_avroschema/fields.py", line 449, in get_avro_type
faust_1  |     "symbols": self.default.symbols,
faust_1  | AttributeError: '_MISSING_TYPE' object has no attribute 'symbols'

Expected behavior An AvroSchema something like:

 {"type": "record", "name": "TestEnum", "fields": [{"name": "OptionalEnum", "type": ["null", {"type": "enum", "name": "OptionalEnum", "symbols": ["Mr", "Mrs", "Miss", "Ms"]}]}], "doc": "TestEnum(OptionalEnum: dataclasses_avroschema.types.Enum = ['Mr', 'Mrs', 'Miss', 'Ms'])"}

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:10 (7 by maintainers)

github_iconTop GitHub Comments

1reaction
aaronkubcommented, Jun 19, 2021

On second thought, I will commit to getting a PR open this weekend. 🙂

Read more comments on GitHub >

github_iconTop Results From Across the Web

Apache Avro Union type - Stack Overflow
Please find below the two schemas. First schema contains union type as "type" : ["null", "string"] and second schema contains union type as...
Read more >
Avro encoding failure producing union types #152 - GitHub
I'm running the REST proxy and schema registry through the official Docker containers with the confluent-platform-2.10.4 package installed, ...
Read more >
Chr.Avro Types and conversions
Unions must contain more than one schema. Avro doesn't explicitly disallow empty unions, but they can't be serialized or deserialized. When mapping a...
Read more >
Schema Registry with union types? - Google Groups
Do union types work with the schema registry and/or the rest proxy? I'd like to use a schema that has optional values, using...
Read more >
Multiple Event types in same topic with unions - Confluent Forum
I am trying to the set up a topic to handle multiple types safely as suggested using schema references. I was able to...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found