question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Failed to create schema with optional fields

See original GitHub issue

Just tried to use schema creation and it’s awesome, but I found, I think, bug with optional fields.

Here is my model:

from typing import Optional
from pydantic import BaseModel

class TestModel(BaseModel):
    foo: Optional[int] = None

print(TestModel.schema_json(indent=2))

Output:

Traceback (most recent call last):
  File "test_pydantic_schema.py", line 9, in <module>
    print(TestModel.schema_json(indent=2))
  File "/pydantic/pydantic/main.py", line 288, in schema_json
    return json.dumps(cls.schema(by_alias=by_alias), default=pydantic_encoder, **dumps_kwargs)
  File "/pydantic/pydantic/main.py", line 278, in schema
    s['properties'] = {f.alias: f.schema(by_alias) for f in cls.__fields__.values()}
  File "/pydantic/pydantic/main.py", line 278, in <dictcomp>
    s['properties'] = {f.alias: f.schema(by_alias) for f in cls.__fields__.values()}
  File "/pydantic/pydantic/fields.py", line 146, in schema
    if issubclass(self.type_, Enum):
TypeError: issubclass() arg 1 must be a class

Same issue with Union:

from typing import Union
from pydantic import BaseModel

class TestModel(BaseModel):
    foo: Union[str, int]

print(TestModel.schema_json(indent=2))

I think pydantic should cover cases like this, I don’t know what is the best way to solve issue with Optional, but in case of Union oneOf or anyOf rule can be used http://json-schema.org/latest/json-schema-validation.html#rfc.section.6.7.

Issue Analytics

  • State:closed
  • Created 5 years ago
  • Comments:14 (13 by maintainers)

github_iconTop GitHub Comments

2reactions
samuelcolvincommented, Jul 2, 2018

as per discussion on gitter:

  • the top level property should change type > schema
  • there should be a shortened display of just 'schema': {'type': 'int'} which is just 'schema': 'int', this applies both at the top level for simple types and where simple types are nested within more complex types.

Overal the design is encapsulated by:

'foobar': {
  'schema': {
    'type': 'list',
    'item_type': {
      'mode': 'any_of',
      'types': [
        'float', 'int',
      ],
    }
  },
  'name': 'Foobar',
  'required': True,
}
1reaction
samuelcolvincommented, Jul 2, 2018

ok, after looking at #218 I propose the following way to deal with this:

type becomes a dictionary.

For “simple” fields, eg. just an int or float, type like:

'type': {
  'item_type': 'float',
}

For unions we’d have:

'type': {
  'mode': 'any_of',
  'item_type': ['str', 'float'],
}

For things like List[int] we’d have:

'type': {
  'item_type': 'int',
  'shape': 'list',
}

or List[Union[int, str]] we’d have:

'type': {
  'mode': 'any_of',
  'item_type': ['int', 'str'],
  'shape': 'list',
}

For Dict[str, int] we’d have:

'type': {
  'item_type': 'int',
  'key_type': 'str',
  'shape': 'dict',
}

(we could add key_mode if the key is a union, but that’s a very rare case)

The names mode, item_type and shape are not great (suggestions very welcome), but I think this is the best way to go.

Questions:

  • are their any case’s I’ve missed?
  • do we do type: {'item_type': 'int', ...} or just add all these items to the main dict?
  • what do we do when implement tuples #12, I guess something like properties on sub models?
  • what do we do with Union[int, FoobarModel] - I guess we have type as above, then properties. What do we do with Union[FooModel, BarModel] - maybe this is just banned?

@Gr1N @x1ah

Read more comments on GitHub >

github_iconTop Results From Across the Web

Failed to create schema with optional fields #213 - GitHub
Just tried to use schema creation and it's awesome, but I found, I think, bug with optional fields. Here is my model: from...
Read more >
Issues with optional fields when using Avro schema
I'm trying to create a schema with optional fields. Here's my setup: $> cat mytest.avsc { "type": "record", "namespace": "com.example", ...
Read more >
Is it possible to have an optional field in an Avro schema (i.e. ...
JSON file)? In my Avro schema, I have two fields: {"name": "author", "type": ["null", "string"], "default": null}, {"name": "importance", "type ...
Read more >
Adding optional field to existing schema, RDBLoader failure
Hi, I am facing the following issue. I want to add an additional optional field to an existing schema. In the schema file...
Read more >
Kafka-Rest Avro schema optional field error - Google Groups
I am using the kafka-rest-proxy to write out to the schema registry and kafka, but I am getting an error as below: "Avro...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found