[BUG] Use third party class as property in pydantic schema

Describe the bug I have a pydantic schema that needs a third party class (bson.objectid.ObjectID) as a property. For this reason I created a custom validator and encoder as per pydantic documentation.

Code

from bson.objectid import ObjectId
from pydantic import BaseModel
from pydantic import validators
from pydantic.errors import PydanticTypeError
from pydantic.utils import change_exception

class ObjectIdError(PydanticTypeError):
    msg_template = 'value is not a valid bson.objectid.ObjectId'

def object_id_validator(v) -> ObjectId:
    with change_exception(ObjectIdError, ValueError):
        v = ObjectId(v)
    return v

def get_validators() -> None:
    yield validators.not_none_validator
    yield object_id_validator

ObjectId.__get_validators__ = get_validators

def encode_object_id(object_id: ObjectId):
    return str(object_id)

class UserId(BaseModel):
    object_id: ObjectId = None

    class Config:
        json_encoders = {
            ObjectId: encode_object_id
        }

class User(UserId):
    email: str
    salt: str
    hashed_password: str

# Just for testing
user = User(object_id = ObjectId(), email="[e-mail]", salt="12345678", hashed_password="letmein")
print(user.json())
# Outputs:
# {"object_id": "5c7e424225e2971c8c548a86", "email": "[e-mail]", "salt": "12345678", "hashed_password": "letmein"}

As you can see at the bottom of the code, the serialization seems to work just fine. But when I use this schema as an argument (and/or response type) in API operations and then open the automatic documentation, I get presented with an error.

Code

from bson import ObjectId
from fastapi import FastAPI
from user import User, UserId

app = FastAPI()


@app.post("/user", tags=["user"], response_model=UserId)
def create_user(user: User):
    # Create user and return id
    print(user)
    return UserId(objectId=ObjectId())

Log

INFO: ('127.0.0.1', 2706) - "GET /openapi.json HTTP/1.1" 500
ERROR: Exception in ASGI application
Traceback (most recent call last):
  File "<project-path>\venv\lib\site-packages\uvicorn\protocols\http\h11_impl.py", line 373, in run_asgi
    result = await asgi(self.receive, self.send)
  File "<project-path>\venv\lib\site-packages\uvicorn\middleware\debug.py", line 83, in __call__
    raise exc from None
  File "<project-path>\venv\lib\site-packages\uvicorn\middleware\debug.py", line 80, in __call__
    await asgi(receive, self.send)
  File "<project-path>\venv\lib\site-packages\starlette\middleware\errors.py", line 125, in asgi
    raise exc from None
  File "<project-path>\venv\lib\site-packages\starlette\middleware\errors.py", line 103, in asgi
    await asgi(receive, _send)
  File "<project-path>\venv\lib\site-packages\starlette\exceptions.py", line 74, in app
    raise exc from None
  File "<project-path>\venv\lib\site-packages\starlette\exceptions.py", line 63, in app
    await instance(receive, sender)
  File "<project-path>\venv\lib\site-packages\starlette\routing.py", line 43, in awaitable
    response = await run_in_threadpool(func, request)
  File "<project-path>\venv\lib\site-packages\starlette\concurrency.py", line 24, in run_in_threadpool
    return await loop.run_in_executor(None, func, *args)
  File "C:\Program Files (x86)\Python37-32\lib\concurrent\futures\thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "<project-path>\venv\lib\site-packages\fastapi\applications.py", line 83, in <lambda>
    lambda req: JSONResponse(self.openapi()),
  File "<project-path>\venv\lib\site-packages\fastapi\applications.py", line 75, in openapi
    openapi_prefix=self.openapi_prefix,
  File "<project-path>\venv\lib\site-packages\fastapi\openapi\utils.py", line 230, in get_openapi
    flat_models=flat_models, model_name_map=model_name_map
  File "<project-path>\venv\lib\site-packages\fastapi\utils.py", line 45, in get_model_definitions
    model, model_name_map=model_name_map, ref_prefix=REF_PREFIX
  File "<project-path>\venv\lib\site-packages\pydantic\schema.py", line 461, in model_process_schema
    model, by_alias=by_alias, model_name_map=model_name_map, ref_prefix=ref_prefix
  File "<project-path>\venv\lib\site-packages\pydantic\schema.py", line 482, in model_type_schema
    f, by_alias=by_alias, model_name_map=model_name_map, ref_prefix=ref_prefix
  File "<project-path>\venv\lib\site-packages\pydantic\schema.py", line 238, in field_schema
    ref_prefix=ref_prefix,
  File "<project-path>\venv\lib\site-packages\pydantic\schema.py", line 440, in field_type_schema
    ref_prefix=ref_prefix,
  File "<project-path>\venv\lib\site-packages\pydantic\schema.py", line 643, in field_singleton_schema
    raise ValueError(f'Value not declarable with JSON Schema, field: {field}')
ValueError: Value not declarable with JSON Schema, field: object_id type=ObjectId default=None

To Reproduce Copy my code and follow the instrcutions given in the “Describe the bug” section.

Expected behavior No error should occur and the documentation should be able to show the schema correctly.

Environment:

• OS: Windows 10
• FastAPI: 0.6.3
• Python: 3.7.2

[stefanondisponibile]

I’m using the solution proposed by @tiangolo up above, just I preferred doing this:

class ObjectIdStr(str):
    @classmethod
    def __get_validators__(cls):
        yield cls.validate

    @classmethod
    def validate(cls, v):
        try:
            ObjectId(str(v))
        except InvalidId:
            raise ValuerError("Not a valid ObjectId")
        return str(v)

this way I can either pass a valid ObjectId string or an ObjectId instance.

This works pretty nicely also with mongoengine, as you’ll be able to pass that ObjectIdStr directly to the db_model, and it will convert the ObjectIdStrings to actual ObjectIds in Mongo.

What I’m striving to understand now, though, is why can’t I get an ObjectId back from the jsonable_encoder by setting this in Config’s json_encoders property:

from bson import ObjectId
from pydantic import BaseModel
from upabove import ObjectIdStr

class SomeItem(BaseModel):
    some_id: ObjectIdStr
   
    class Config:
        json_encoders = {ObjectIdStr: lambda x: ObjectId(x)}

Why wouldn’t some_id be converted to an ObjectId when calling jsonable_encoder on SomeItem instance? Is it maybe because being some_id a str it won’t be passed further down to the custom json_encoders? This even if ObjectIds, are not json serializable.

[tiangolo]

About https://github.com/samuelcolvin/pydantic/pull/520, it was superseded by https://github.com/samuelcolvin/pydantic/pull/562.

While reviewing it I tested with bson, and I realized that it doesn’t necessarily fix the problem, but that you can fix it like this:

from bson import ObjectId
from pydantic import BaseModel


class ObjectIdStr(str):
    @classmethod
    def __get_validators__(cls):
        yield cls.validate

    @classmethod
    def validate(cls, v):
        if not isinstance(v, ObjectId):
            raise ValueError("Not a valid ObjectId")
        return str(v)


class UserId(BaseModel):
    object_id: ObjectIdStr = None


class User(UserId):
    email: str
    salt: str
    hashed_password: str

# Just for testing
user = User(object_id = ObjectId(), email="[e-mail]", salt="12345678", hashed_password="letmein")
print(user.json())
# Outputs:
# {"object_id": "5c7e424225e2971c8c548a86", "email": "[e-mail]", "salt": "12345678", "hashed_password": "letmein"}

The trick is, there’s no way to declare a JSON Schema for a BSON ObjectId, but you can create a custom type that inherits from a str, so it will be declarable in JSON Schema, and it can take an ObjectId as input.

Then, if you need the ObjectId itself (instead of the str version), you can create another model that has the ObjectId as you declared it before, and copy the values from the input/to the output.