multipart/form-data: Unable to parse complex types in a request form

First check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn’t find it. - See additional context
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google “How to X in FastAPI” and didn’t find any information.
• I already read and followed all the tutorial in the docs and didn’t find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.
• After submitting this, I commit to one of:
  • Read open issues with questions until I find 2 issues where I can help someone and add a comment to help there.
  • I already hit the “watch” button in this repository to receive notifications and I commit to help at least 2 people that ask questions in the future.
  • Implement a Pull Request for a confirmed bug.

Example

Here’s a self-contained, minimal, reproducible, example with my use case:

import inspect
from typing import Dict, Type

from fastapi import Depends, FastAPI, File, Form
from pydantic import BaseModel

app = FastAPI()


def as_form(cls: Type[BaseModel]):
    """
    Adds an as_form class method to decorated models. The as_form class method
    can be used with FastAPI endpoints
    """
    new_params = [
        inspect.Parameter(
            field.alias,
            inspect.Parameter.POSITIONAL_ONLY,
            default=(Form(field.default) if not field.required else Form(...)),
            annotation=field.outer_type_,
        )
        for field in cls.__fields__.values()
    ]

    async def _as_form(**data):
        return cls(**data)

    sig = inspect.signature(_as_form)
    sig = sig.replace(parameters=new_params)
    _as_form.__signature__ = sig
    setattr(cls, "as_form", _as_form)
    return cls


@as_form
class Item(BaseModel):
    name: str
    another: str
    opts: Dict[str, int] = {}


@app.post("/test", response_model=Item)
def endpoint(item: Item = Depends(Item.as_form), data: bytes = File(...)):
    print(len(data))
    return item


if __name__ == "__main__":
    import json
    import os

    from fastapi.testclient import TestClient

    tc = TestClient(app)

    item = {"name": "vivalldi", "another": "mause"}

    data = bytearray(os.urandom(1))
    files = {"data": ("data", data, "text/csv")}

    r = tc.post("/test", data=item, files=files)
    assert r.status_code == 200
    assert r.json() == {"name": "vivalldi", "another": "mause", "opts": {}}

    files["opts"] = (None, json.dumps({"a": 2}), "application/json")
    r = tc.post("/test", data=item, files=files)
    assert r.status_code == 200
    assert r.json() == {"name": "vivalldi", "another": "mause", "opts": {"a": 2}}

Description

tl;dr

Complex data types (objects) are not supported in multipart/form-data by FastAPI. There are workarounds for top level fields in pydantic models; nested objects are not supported. Save the above script as nested.py. Run python ./nested.py to see failing assertions. And call uvicorn nested:app to run the FastAPI app locally. Local tests can be run using httpie script in this gist

full

If you look through the StackOverflow & FastAPI issues you’ll find plenty of examples of how to convert a model into a form. This issue is to address the shortcomings of those workarounds. The main focus of this is to determine the best way to work with multi-content multipart/form-data requests.

Per the OpenAPI Special Considerations for multipart Content “boundaries MAY be used to separate sections of the content being transferred.” This indicates that you can specify the content type of each individual part. An example of a combination multipart request can be found in this gist. The gist was generated using httpie.

Going forward I intend to investigate what can be done from a workaround standpoint. Initially, I think we can adjust the workarounds to set object types to a File and parse those files as JSON (might require some model tweaks as well). Additionally, if this issue gains traction I will look into making changes that allow FastAPI to better support multi-content multipart requests.

Environment

• OS: macOS
• FastAPI Version: 0.58.1
• Python version: 3.8.6

Additional context

This is a spin-off of #2365 at Mause’s request

Since the root cause is complex types (objects) in forms aren’t supported, this may be a duplicate of #2295

[sam-kleiner]

Hi, I am just wondering why not add something like this. The error(type is not dict) is the value of Item comes in as string when it is multipart/form-data. This solution also works with nested classes, it also won’t unpack all fields of Item into form fields, which is not very nice if this Item class is a little complicated. Happy to create a PR if this is really a good solution.

For anyone else who needs nested models, I made this alternate approach that takes the input in the same way @tricosmo takes in the value as json. This method does do the proper validation! I really needed to take a file and complex form field and this gets the job done. It should also support multiple form properties.

fastapi code:

class TestMetaConfig(BaseModel):
    prop1: str
    prop2: list[int]


class TestModel(BaseModel):
    foo: str
    bar: int
    meta_config: TestMetaConfig


@app.post("/")
async def foo(upload_file: UploadFile = File(...), model: Json[TestModel] = Form(...)):
    with open("test.png", "wb") as fh:
        fh.write(await upload_file.read())

    logger.info(f"model.foo: {model.foo}")
    logger.info(f"model.bar: {model.bar}")
    logger.info(f"model.meta_config: {model.meta_config}")
    return {"foo": model.foo, "bar": model.bar}

Call it like this with json dumps

import json
from pathlib import Path

import requests

HERE = Path(__file__).parent.absolute()

with open(HERE / "imgs/foo.png", "rb") as fh:
    url = "http://localhost:8000/"
    files = {"upload_file": fh}
    values = {"foo": "hello", "bar": 123, "meta_config": {"prop1": "hello there", "prop2": ["general", "kenobi", 1]}}
    resp = requests.post(url, files=files, data={"model": json.dumps(values)})
    print(resp.status_code)
    print(resp.json())

Proper validation if you send in bad types like I did in my example above

{"detail": [{"loc": ["body", "model", "meta_config", "prop2", 0], "msg": "value is not a valid integer", "type": "type_error.integer"}, {"loc": ["body", "model", "meta_config", "prop2", 1], "msg": "value is not a valid integer", "type": "type_error.integer"}]}

[Mause]

Try this:

import inspect
from typing import Dict, Type, TypeVar, Protocol, Generic, NewType

from fastapi import Depends, FastAPI, File, Form
from pydantic import BaseModel, validator, BaseSettings, Json

app = FastAPI()

StringId = NewType('StringId', str)


def as_form(cls: Type[BaseModel]):
    """
    Adds an as_form class method to decorated models. The as_form class method
    can be used with FastAPI endpoints
    """
    new_params = [
        inspect.Parameter(
            field.alias,
            inspect.Parameter.POSITIONAL_ONLY,
            default=(Form(field.default) if not field.required else Form(...)),
        )
        for field in cls.__fields__.values()
    ]

    async def _as_form(**data):
        return cls(**data)

    sig = inspect.signature(_as_form)
    sig = sig.replace(parameters=new_params)
    _as_form.__signature__ = sig
    setattr(cls, "as_form", _as_form)
    return cls


@as_form
class Item(BaseModel):
    name: str
    another: str
    opts: Json[Dict[str, int]] = '{}'


@app.post("/test")
async def endpoint(item: Item = Depends(Item.as_form)):
    return item.dict()


if __name__ == "__main__":
    import json
    import os

    from fastapi.testclient import TestClient

    tc = TestClient(app)

    item = {"name": "vivalldi", "another": "mause"}

    data = bytearray(os.urandom(1))
    files = {"data": ("data", data, "text/csv")}

    r = tc.post("/test", data=item, files=files)
    assert r.status_code == 200, r.text
    assert r.json() == {"name": "vivalldi", "another": "mause", "opts": {}}

    files["opts"] = (None, json.dumps({"a": 2}), "application/json")
    r = tc.post("/test", data=item, files=files)
    assert r.status_code == 200
    assert r.json() == {"name": "vivalldi", "another": "mause", "opts": {"a": 2}}

The main two changes were using the Json class from pydantic, and removing the annotation from the as_form method, as otherwise pydantic would be validating the data twice - once for as_form, and once for the model itself