question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Recursive loading with schema.load_schema is failing in 1.2.1

See original GitHub issue

Using the example at https://fastavro.readthedocs.io/en/latest/schema.html (but adding

    "namespace": "namespace"

to Parent.avsc), recursive loading works for fastavro 1.1.1, but fails for 1.2.1. (Py3.9, installed with homebrew on MacOS 10.14.6, in a venv)

See the comparison below (1.1.1 top; 1.2.1 bottom). Thanks

In [1]: import fastavro

In [2]: ??fastavro.__version__
Type:        str
String form: 1.1.1
Length:      5
Docstring:  
str(object='') -> str
str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or
errors is specified, then the object must expose a data buffer
that will be decoded using the given encoding and error handler.
Otherwise, returns the result of object.__str__() (if defined)
or repr(object).
encoding defaults to sys.getdefaultencoding().
errors defaults to 'strict'.

In [3]: fastavro.schema.load_schema("Parent.avsc");

In [4]: fastavro.schema.load_schema("Parent.avsc")
Out[4]: 
[{'type': 'record',
  'name': 'namespace.Child',
  'fields': [],
  '__fastavro_parsed': True,
  '__named_schemas': {'namespace.Child': {'type': 'record',
    'name': 'namespace.Child',
    'fields': []},
   'namespace.Parent': {'type': 'record',
    'name': 'namespace.Parent',
    'fields': [{'name': 'child', 'type': 'namespace.Child'}]}}},
 {'type': 'record',
  'name': 'namespace.Parent',
  'fields': [{'name': 'child', 'type': 'namespace.Child'}],
  '__fastavro_parsed': True,
  '__named_schemas': {'namespace.Child': {'type': 'record',
    'name': 'namespace.Child',
    'fields': []},
   'namespace.Parent': {'type': 'record',
    'name': 'namespace.Parent',
    'fields': [{'name': 'child', 'type': 'namespace.Child'}]}}}]

In [5]: quit
Python 3.9.0 (default, Dec  6 2020, 20:20:21) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.19.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import fastavro

In [2]: ??fastavro.__version__
Type:        str
String form: 1.2.1
Length:      5
Docstring:  
str(object='') -> str
str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or
errors is specified, then the object must expose a data buffer
that will be decoded using the given encoding and error handler.
Otherwise, returns the result of object.__str__() (if defined)
or repr(object).
encoding defaults to sys.getdefaultencoding().
errors defaults to 'strict'.

In [3]: fastavro.schema.load_schema("Parent.avsc");
---------------------------------------------------------------------------
UnknownType                               Traceback (most recent call last)
fastavro/_schema.pyx in fastavro._schema._load_schema()

fastavro/_schema.pyx in fastavro._schema.parse_schema()

fastavro/_schema.pyx in fastavro._schema._parse_schema()

fastavro/_schema.pyx in fastavro._schema.parse_field()

fastavro/_schema.pyx in fastavro._schema._parse_schema()

UnknownType: namespace.Child

During handling of the above exception, another exception occurred:

Exception                                 Traceback (most recent call last)
<ipython-input-3-8c930452e82a> in <module>
----> 1 fastavro.schema.load_schema("Parent.avsc");

fastavro/_schema.pyx in fastavro._schema.load_schema()

fastavro/_schema.pyx in fastavro._schema._load_schema()

fastavro/_schema.pyx in fastavro._schema._inject_schema()

fastavro/_schema.pyx in fastavro._schema._inject_schema()

Exception: Internal error; You should raise an issue in the fastavro github repository

In [4]: quit

Issue Analytics

  • State:closed
  • Created 3 years ago
  • Comments:6

github_iconTop GitHub Comments

1reaction
scottbeldencommented, Dec 24, 2020

@majensen Thanks again for the example. I think I have it fixed in PR #492. If you have a moment, could you test using that code? It works for your example but I just wanted to see if there were any other issues that your schema runs into.

To test you should be able to grab the wheel here and then install it with something like this:

pip uninstall fastavro
pip install --no-index -f <download_folder> fastavro

If you don’t have time to test then that’s okay. I’ll still plan to merge and release it sometime soon as it definitely is a bug.

1reaction
scottbeldencommented, Dec 22, 2020

Thanks for the report. I need to fix that example as it fails currently even without adding the namespace field to Parent. However, it looks like it should be pretty easy to resolve the issue you are seeing. I’ll try to take a look soon but the holidays might delay it a bit.

Read more comments on GitHub >

github_iconTop Results From Across the Web

Unable to use recursive schema · Issue #1491 - GitHub
Describe the bug I have a json schema which uses a $ref as a items ... A schema with recursion should work, or...
Read more >
Cannot validate recursive schema - json - Stack Overflow
Only one of the schemas can validate to true. If both properties are present, the schema will fail. Next, let's define a file...
Read more >
Stardog Platform Release Notes
... Introduce a db option to set the dataset for GraphQL queries (#PLAT-4413); Data catalog automatically create database and load schema if not...
Read more >
Changelog - Gentics Mesh
GraphQL: all requests to graphql v2 were failing if one of the schemas included an s3 binary field. This has been fixed. Dependencies:...
Read more >
ajv - NPM Package Overview - Socket - Socket.dev
In this case Ajv can either fail schema compilation (default) or ignore it ... function loadSchema(uri) { return request.json(uri).then(function (res) { if ...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found