Enforced data size in channels 2.1.7 prevent large file uploads

Channels was released a few days ago (kudos and thank you for the maintainance effort!) and we noticed a breaking change in our application.

Large file uploads that were working before are now failing, and understand it’s related to this specific commit https://github.com/django/channels/commit/a1ecd5ee72a538f19bdd9e8f6bb91bb8aabba5d9 by @Zarathustra2

I’m not discussing the rationale behind that commit, because enforcing the max size looks indeed better than before from a security/stability perspective. However, it is a breaking change for our app (and possibly others), because file uploads that were working before are now failing.

Since we were not setting DATA_UPLOAD_MAX_MEMORY_SIZE in our settings file, the default value (2.5Mio) applies, which is considerably lower than our average uploaded file size.

In theory, we could stick on channels==2.1.7 and set DATA_UPLOAD_MAX_MEMORY_SIZE to match our requirements, but AFAIU, it would apply to the whole payload, and not only file size. And we’d like to allow large files, but not excessively large POST data.

Also, as stated in Django’s documentation about DATA_UPLOAD_MAX_MEMORY_SIZE:

The check is done when accessing request.body or request.POST and is calculated against the total request size excluding any file upload data.`

Based on that, I do think there is a bug in the way the check was implemented in channels, because it does not exclude file data.

I can try working on a PR if you agree with my suggested changes:

• Document the breaking change in the changelog, so people know what to do when upgrading
• Remove uploaded files when computed payload size

Let me know your thoughts and I’ll start working on a fix 😃

[carltongibson]

Fixed in #1352. Will be out later today.

[carltongibson]

Hi @pythonBerg. This is just waiting for a bit of bandwidth over the summer to finish off.

The PR in #1251 works. You can install that and give it a run and report back there. That would be super.

As it stands I’m not ready to just push it out to folks as maybe there’s hidden issues. It’s delicate. I want to test it first.

So, in its place I want to add a SpooledTemporaryFile version, that is much more clearly safe, and then offer the experimental version as an option for the brave to try. Once we’ve had a chance to see it in action, it can become the default. (In theory it’s the way to go…)

The hold up is just person-power. A set of releases here for Channels, Daphne and Channels redis is my goal for the summer. Any input from fellow humans here or elsewhere is greatly appreciated.