FastAPI ignores json_encoders in the Model

First check

• [*] I added a very descriptive title to this issue.
• [*] I used the GitHub search to find a similar issue and didn’t find it.
• [*] I searched the FastAPI documentation, with the integrated search.
• [*] I already searched in Google “How to X in FastAPI” and didn’t find any information.
• [*] I already read and followed all the tutorial in the docs and didn’t find an answer.
• [*] I already checked if it is not related to FastAPI but to Pydantic.
• [*] I already checked if it is not related to FastAPI but to Swagger UI.
• [*] I already checked if it is not related to FastAPI but to ReDoc.
• [*] After submitting this, I commit to one of:
  • Read open issues with questions until I find 2 issues where I can help someone and add a comment to help there.
  • I already hit the “watch” button in this repository to receive notifications and I commit to help at least 2 people that ask questions in the future.
  • Implement a Pull Request for a confirmed bug.

Example

Below is an example that showcases the issue:

from typing import *

from pydantic import BaseModel
from fastapi.encoders import jsonable_encoder

def sort_func(x: Set[str]) -> List[str]:
   """This function should be called on every serialization. This will convert a Set of strings to a sorted List of strings."""
    print("i was called!")
    return sorted(x)


class SomeModel(BaseModel):
    stuff: Set[str] = ...

    class Config:
        """Define a custom encoder for Sets such that they are sorted lists."""
        json_encoders = {
            set: sort_func
        }

print(SomeModel(stuff={"z", "a", "f", "d"}).json())
# This prints:
# "i was called!"  # GOOD
# '{"stuff": ["a", "d", "f", "z"]}'  # GOOD
# The Pydandic serializer does what it's supposed to do.

print(jsonable_encoder(SomeModel(stuff={"z", "a", "f", "d"})))
# This prints:
# {'stuff': ['z', 'd', 'f', 'a']}  # BAD
# The fastapi encoder is not calling the `sort_func`

Description

• The primary issue is that when a Pydantic model defines special serialization behavior, fastapi is not obeying it.
• In my example, I’m trying to store a set of values internally, but then serialize it into a sorted list.
• The desired behavior is for fastapi to be consistent and obey Pydantic’s conventions.

I also looked at other issues that are similar, but not quite the same:

• https://github.com/tiangolo/fastapi/issues/1722
• https://github.com/tiangolo/fastapi/issues/1986

Environment

• OS: macOS
• FastAPI Version: fastapi==0.61.1
• Python version: 3.8.5

Additional context

I’m not sure if this is related but I think this is also a bug in the jsonable_encoder:

def jsonable_encoder(
    obj: Any,
    include: Optional[Union[SetIntStr, DictIntStrAny]] = None,
    exclude: Optional[Union[SetIntStr, DictIntStrAny]] = None,
    by_alias: bool = True,
    exclude_unset: bool = False,
    exclude_defaults: bool = False,
    exclude_none: bool = False,
    custom_encoder: dict = {},  # <------- 
    sqlalchemy_safe: bool = True,
) -> Any:

https://github.com/tiangolo/fastapi/blob/0dfde6e284b221bf6695a98762a56d13a995e807/fastapi/encoders.py#L34 The custom_encoder is a keyword argument that defines a mutable type.

[mdgilene]

I’m having troubles with this fragment here:

https://github.com/tiangolo/fastapi/blob/fdb6c9ccc504f90afd0fbcec53f3ea0bfebc261a/fastapi/encoders.py#L43-L46

In my case, I’m attempting to temporarily apply encoding logic for a pydantic object but I very much don’t want it to be permanent on the model’s Config.json_encoders dict. If line 44 would instead be something like:

encoder = dict(getattr(obj.__config__, "json_encoders", {}))

So as to copy the original encoders dict, so the permanent config mutation doesn’t occur. I’m not sure this is a bug strictly, just an expectation issue: the jsonable_encoder should not ever mutate a pydantic class. Then again, I think the custom_encoder parameter is not even documented and maybe it’s not intended for general use.

EDIT: I’ll leave my current workaround here: instead of passing a pydantic object to jsonable_encoder, I call the object’s dict() method beforehand:

# assume obj is a pydantic object
# assume custom_encoder contains the custom encoders to be given for this encoding only
encoded = jsonable_encoder(obj.dict(), custom_encoder=dict(obj.__config__.json_encoders, **custom_encoder))

I just ran into this exact same issue. jsonable_encoder modifying the classes config has to be a bug. If not, this is EXTREMELY unintuitive behavior and should be changed.

[mikegrima]

To help close the loop, if you wanted to have sorted JSON results back, you would need to have something that looks like this:

import json
import typing
from starlette.responses import Response
from fastapi import APIRouter
from pydantic import BaseModel

class SortedJSONResponse(Response):
    media_type = "application/json"

    def render(self, content: typing.Any) -> bytes:
        return json.dumps(
            content,
            ensure_ascii=False,
            allow_nan=False,
            indent=None,
            separators=(",", ":"),
            sort_keys=True   # <---
        ).encode("utf-8")

class SomeModel(BaseModel):
   some_value: int
   some_other_value: str
   and_something_else: bool

some_router = APIRouter()

@some_router.get("/some/api/path", response_class=SortedJSONResponse)
async def get_sorted_json_response_api() -> SomeModel:
   return SomeModel(some_value: 1, some_other_value: "some other value", and_something_else: True)