UploadFile hogs RAM

Good day,

We are using FastAPI to access large (1GB) files through starlette UploadFile and we have noticed that all the file is loaded into RAM.

Looking at the code I see that starlette uses SpooledTemporaryFile, however it initializes it [1] with default parameters, namely max_size=0. The thing is that SpooledTemporaryFile.write() methods calls self._check() to switch to disk IO, however with max_size=0 it never does it [2] and hence the whole file is loaded to RAM.

I suggest to starlette’s UploadFile to:

• Set positive max_size by default
• Accept user-defined value from constructor

What do you think?

[1] https://github.com/encode/starlette/blob/594b95c1da8c4ed2aff5629ab5ce128a18b6931b/starlette/datastructures.py#L428

[2] https://github.com/python/cpython/blob/master/Lib/tempfile.py#L650

[tomchristie]

Flask (werkzeug) uses 500k by default it seems

Okay, so 1MB would be a good midway point between that and Django’s default.

Will you accept a PR without test for this?

Yup for sure - it’s just exercising a standard bit of Python stdlib, so not an issue at all.

[haizaar]

PR ready.