Errors while validating arguments in headers result in a flask crash

If you make a view with header arguments @bp.arguments(someschema, location='headers') Then feed it headers that are not defined in the schema, it will (rightfully) cause a schema validation error, however the error created includes the entire header tuple as a dictionary key, instead of just the ‘key’ (tuple position 0). This causes flask to error out while trying to convert the response to a valid JSON response.

This is the response returned (grabbed this with a pydb)

{
    "code": 422,
    "status": "Unprocessable Entity",
    "errors": {"headers": {("Someheader", "someval"): ["Unknown field."]}},
}

This is the stack trace flask produces, which I have included so people searching it will hopefully find their way here.

../../../../miniconda3/lib/python3.7/site-packages/werkzeug/test.py:1006: in get
    return self.open(*args, **kw)
nomitall/api/tests/_client.py:37: in open
    return super().open(*args, **kwargs)
../../../../miniconda3/lib/python3.7/site-packages/flask/testing.py:227: in open
    follow_redirects=follow_redirects,
../../../../miniconda3/lib/python3.7/site-packages/werkzeug/test.py:970: in open
    response = self.run_wsgi_app(environ.copy(), buffered=buffered)
../../../../miniconda3/lib/python3.7/site-packages/werkzeug/test.py:861: in run_wsgi_app
    rv = run_wsgi_app(self.application, environ, buffered=buffered)
../../../../miniconda3/lib/python3.7/site-packages/werkzeug/test.py:1096: in run_wsgi_app
    app_rv = app(environ, start_response)
../../../../miniconda3/lib/python3.7/site-packages/flask/app.py:2463: in __call__
    return self.wsgi_app(environ, start_response)
../../../../miniconda3/lib/python3.7/site-packages/flask/app.py:2449: in wsgi_app
    response = self.handle_exception(e)
../../../../miniconda3/lib/python3.7/site-packages/flask/app.py:1866: in handle_exception
    reraise(exc_type, exc_value, tb)
../../../../miniconda3/lib/python3.7/site-packages/flask/_compat.py:39: in reraise
    raise value
../../../../miniconda3/lib/python3.7/site-packages/flask/app.py:2446: in wsgi_app
    response = self.full_dispatch_request()
../../../../miniconda3/lib/python3.7/site-packages/flask/app.py:1952: in full_dispatch_request
    return self.finalize_request(rv)
../../../../miniconda3/lib/python3.7/site-packages/flask/app.py:1967: in finalize_request
    response = self.make_response(rv)
../../../../miniconda3/lib/python3.7/site-packages/flask/app.py:2111: in make_response
    rv = jsonify(rv)
../../../../miniconda3/lib/python3.7/site-packages/flask/json/__init__.py:370: in jsonify
    dumps(data, indent=indent, separators=separators) + "\n",
../../../../miniconda3/lib/python3.7/site-packages/flask/json/__init__.py:211: in dumps
    rv = _json.dumps(obj, **kwargs)
../../../../miniconda3/lib/python3.7/site-packages/simplejson/__init__.py:412: in dumps
    **kw).encode(obj)
../../../../miniconda3/lib/python3.7/site-packages/simplejson/encoder.py:298: in encode
    chunks = list(chunks)
../../../../miniconda3/lib/python3.7/site-packages/simplejson/encoder.py:696: in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
../../../../miniconda3/lib/python3.7/site-packages/simplejson/encoder.py:652: in _iterencode_dict
    for chunk in chunks:
../../../../miniconda3/lib/python3.7/site-packages/simplejson/encoder.py:652: in _iterencode_dict
    for chunk in chunks:
../../../../miniconda3/lib/python3.7/site-packages/simplejson/encoder.py:598: in _iterencode_dict
    k = _stringify_key(k)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

key = ("Someheader", "someval")

    def _stringify_key(key):
        if isinstance(key, string_types): # pragma: no cover
            pass
        elif _PY3 and isinstance(key, bytes) and _encoding is not None:
            key = str(key, _encoding)
        elif isinstance(key, float):
            key = _floatstr(key)
        elif key is True:
            key = 'true'
        elif key is False:
            key = 'false'
        elif key is None:
            key = 'null'
        elif isinstance(key, integer_types):
            if type(key) not in integer_types:
                # See marshmallow-code/flask-smorest#118, do not trust custom str/repr
                key = int(key)
            key = str(key)
        elif _use_decimal and isinstance(key, Decimal):
            key = str(key)
        elif _skipkeys:
            key = None
        else:
            raise TypeError('keys must be str, int, float, bool or None, '
>                           'not %s' % key.__class__.__name__)
E           TypeError: keys must be str, int, float, bool or None, not tuple

../../../../miniconda3/lib/python3.7/site-packages/simplejson/encoder.py:568: TypeError

I’m not sure there is a good workaround apart from disabling schema validation.

If this issue is unclear please ask for further explanation and i’ll sink some time into creating some self contained reproduction code.

[sirosen]

I just got some time to work on this. I’m pretty sure the issue is that marshmallow is calling set(MultiDictProxy), expecting that to produce the set of dict keys which it will treat as fieldnames.

MultiDictProxy calls iter(request.headers), and this is the result.

So I think the solution might be for MultiDictProxy to get more intelligent about how it exposes iter. I’m fiddling with this now. Hope to have something reviewable soon.

[shughes-uk]

Might be worth a small line in the documentation about that, I got caught by it and this error made tracing the source much more difficult.