Busboy with AWS Lambda function

Hi Everyone,

I am trying to use busboy to parse multipart/form-data coming from AWS API Gateway to AWS Lambda. I have used LAMBDA_PROXY integration.

My parser is as follows:

export function parseMultipartFormData(event) {
  return new Promise((resolve, reject) => {
    const busboy = new Busboy({
      headers: {
        'content-type': event.headers['Content-Type'],
      },
    });

    const body = {};
    let buffer = '';

    busboy.on('file', (fieldname, file, filename, encoding, mimetype) => {
      body.filename = filename;
      body.encoding = encoding;
      body.mimetype = mimetype;

      file.setEncoding('base64');

      file.on('data', (chunk) => {
        buffer += chunk;
      });

      file.once('end', () => {
        body.documentContent = buffer;
      });
    })
      .on('field', (fieldname, val) => {
        body[fieldname] = val;
      })
      .on('finish', () => {
        resolve(body);
      })
      .on('error', (err) => {
        reject(err);
      });

    busboy.end(event.body);
  });
}

Parser is working fine for fields. When I upload a .pdf file it gets corrupted. So I checked the same file’s base_64 encoded string on my local machine using this block of code:

const fs = require('fs');
const { Buffer } = require('buffer');
const path = require('path');

const stream = fs.createReadStream(path.join(__dirname, 'file.pdf'));

let buffer = '';

stream.setEncoding('base64');

stream.on('data', (chunk) => {
  buffer += chunk;
});

stream.on('end', () => {
  console.log(buffer);
});

Then I compared the base_64 encoded string with the string generated by busboy parser and somehow that string is not matching for the same file and encoding.

Is there anything wrong with using busboy with lambda? Or something is wrong in my integration?

[mscdex]

Previously I was using Buffer.concat but it didn’t work

It is the correct way for buffering binary data though and will work for either text or non-text data.

[mscdex]

If event.body is a base64-encoded string, then you need to either convert it to a Buffer first (from base64) and pass that to busboy.end() instead or pass the encoding ('base64') as the second argument to busboy.end() so that Busboy can correctly deal with the actual raw, binary form data.

Also the file varying between S3 and your local machine sounds suspect. Maybe it did get corrupted during upload to S3 after all?