6.0.0 requires changes to some pre_load hooks

Just upgraded to 6.0.0. Using FlaskParser, the input to Schema.load is now a MultiDictProxy wrapping a werkzeug ImmutableMultiDict instead of a regular dictionary. This breaks a schema I have that uses a pre_load hook to converts empty strings to None. (I’m processing an HTML form with a bunch of optional number inputs similar to some of the solutions in marshmallow-code/marshmallow#713)

Not the biggest deal to work around but I thought I would mention it because I didn’t see it in the change log.

Also it would be convenient if MultiDictProxy had a __repr__ to make debugging more convenient.

[sirosen]

I think the issue with pre_load hooks pre 6.0 is that they weren’t used when gathering parameters from various locations as described in #173. The pre_load hook is still called but just called with the wrong data.

Yes, you’re spot on. I had gotten myself mixed up; sorry about that. In 5.x, parsing assembled a new dict of data, which is what got passed to the schema.load method (and thereby, to pre_load and post_load).

That also explains why your hook worked in 5.x, but not in 6.0. In 5.x, you get a dict with your parameters. But the MultiDictProxy you get in 6.0 wraps the raw flask/werkzeug datatype (an ImmutableMultiDict).

I just looked, and werkzeug supports converting an ImmutableMultiDict to a (mutable) MultiDict. So I think for right now you could start your pre_load hook with a line like this

    @pre_load
    def convert_empty_string_to_none(self, data):
        # unfortunate names... makes the underlying datatype mutable
        data.data = data.data.copy()

I’m thinking about whether or not / how webargs could support this more “out of the box”. For example, should we just call copy() on all flask multidicts on their way into the proxy object? (We’d have to check out each framework on this front too.) I don’t want to rush out a solution, but this seems like a usage we need to support – rewriting the contents of the dict in a pre_load hook.

[sirosen]

If you’re comfortable with us asking users (like yourself) to do the conversion, I’m happy with that. The performance penalty is probably negligible in the majority of cases, but I’m sure that for some combination of location/framework it would be an issue. It would be great to avoid it.

Maybe I should just close #491 . I can try to add something reasonable to the docs soon, once the upgrade doc is finished. The location you suggested looks good to me.