Binary data is returned base64 encoded

I am attempting to use Chalice to return a PDF file. It works fine with chalice local, but when live, all the content returned is base64 encoded instead of returning raw bytes. I believe I am following all of the guidelines to get this to work: I am setting the Content-Type header, that type is added to binary_types, and the data I am providing is of type ‘bytes’.

I have built a minimal test case, using a PDF file in /vendors/test.pdf:

• app.py

from chalice import Chalice, Response, NotFoundError

from pprint import pprint

app = Chalice(app_name='pdf-download-test-case')

app.api.binary_types.append('application/pdf')

@app.route('/')
def index():
    with open('./test.pdf', 'rb') as pdf:
        content = pdf.read()

    pprint(type(content))
    pprint(app.api.binary_types)

    return Response(body=content,
                    status_code=200,
                    headers={'Content-Type': 'application/pdf'})

• output (verifying my data is the correct type, and the MIME type was added)

2020-08-28 13:46:13.685000 fea9ad <class 'bytes'>
2020-08-28 13:46:13.685000 fea9ad ['application/octet-stream',
2020-08-28 13:46:13.685000 fea9ad 'application/x-tar',
2020-08-28 13:46:13.685000 fea9ad 'application/zip',
2020-08-28 13:46:13.685000 fea9ad 'audio/basic',
2020-08-28 13:46:13.685000 fea9ad 'audio/ogg',
2020-08-28 13:46:13.685000 fea9ad 'audio/mp4',
2020-08-28 13:46:13.685000 fea9ad 'audio/mpeg',
2020-08-28 13:46:13.685000 fea9ad 'audio/wav',
2020-08-28 13:46:13.685000 fea9ad 'audio/webm',
2020-08-28 13:46:13.685000 fea9ad 'image/png',
2020-08-28 13:46:13.685000 fea9ad 'image/jpg',
2020-08-28 13:46:13.685000 fea9ad 'image/jpeg',
2020-08-28 13:46:13.685000 fea9ad 'image/gif',
2020-08-28 13:46:13.685000 fea9ad 'video/ogg',
2020-08-28 13:46:13.685000 fea9ad 'video/mpeg',
2020-08-28 13:46:13.685000 fea9ad 'video/webm',
2020-08-28 13:46:13.685000 fea9ad 'application/pdf']

Is this a bug, or is there configuration that I am missing?

[msmpublicgoodsw]

Trying to deliver content to browsers is kind of fighting against the way that API Gateway and Chalice were designed.

That’s correct, IMO. API Gateway is, surprise, meant for implementing HTTP APIs for well-behaved API clients. I suspect the more reasonable thing to do – at least for my use case – from a Chalice view function is to respond with an HTTP redirect to binary content on some server that expects to be accessed by browsers, like S3, or even possibly redirect to a data: URL.

[jamesls]

Thanks everyone for the detailed explanations of the current behavior. I think the best options we have at the moment is documenting the behavior with browsers and the options you have for working with HTTP clients that can’t set Accept headers. Also, I think it should be a safe change to support having */* as a binary type and still allow you to return a python dictionary directly instead of requiring you to use the Response(body=json.dumps(...), ...) workaround. Right now, doing so is an error condition, so we should make be able to add a fallback in that case (e.g. try to serialize as utf-8 first if we try to base64 encode a string/unicode type).