Zero bytes file saved with attachFieldsToBody options

🐛 Bug Report

My file saved was zero bites when i use the attachFieldsToBody: true options.

in this case file limit the limit file is also not working properly. It takes 2 minutes for a file size of 90MB to display the message and a message request file too large, please check multipart config

This is my route handler.

const uploadMedia = async (req, res) => {
  const { RecordID, FileName, Extension } = req.body;
  let isUploaded = true;
  const response = {
    uploaded: [],
    rejected: [],
  };

  try {
    const part = await req.body.file
    // const filePath = `uploads/${FileName.value}-${RecordID.value}.${Extension.value}`;
    const filePath = `uploads/${part.filename}`;
    await pump(part.file, fs.createWriteStream(filePath));
    const fileSize = fs.statSync(filePath).size;

    if (part.file.truncated) {
      isUploaded = false;

      // hapus file
      fs.unlinkSync(filePath);
      const rejectedFileDetails = {
        filename: part.filename,
        fileSize,
        reason: `File size too big`
      }
      response.rejected.push(rejectedFileDetails);

      // hapus content media
      await ContentMedia.destroy({
        where: {
          RecordID: RecordID.value
        }
      });
    } else {
      const uploadedFileDetails = {
        filename: part.filename,
        fileSize
      }
      response.uploaded.push(uploadedFileDetails);
    }
  } catch (error) {
    isUploaded = false;
    console.log(error);
    // throw new InternalErrorResponse();
  }

  // sending response
  if (isUploaded) {
    return new SuccessResponse('success', response);
  } else {
    return new UnprocessableEntityResponse(response);
  }
};

This is the register settings

server.register(require('fastify-multipart'), {
  limits: {
    fileSize: nconf.get("maxFileSize"), // 52428800
  },
  attachFieldsToBody: true,
  sharedSchemaId: '#mySharedSchema',
});

This my validation schema

const validateUploadMedia = {
  preValidation: [
    async function (request) {
      return await request.jwtVerify();
    },
  ],
  schema: {
    consumes: ['multipart/form-data'],
    body: {
      type: 'object',
      properties: {
        RecordID: { $ref: '#mySharedSchema' },
        FileName: { $ref: '#mySharedSchema' },
        Extension: { $ref: '#mySharedSchema' },
      },
      required: ['RecordID', 'FileName', 'Extension'],
    },
  },
};

my code was works fine when i use the normal register options only fileSize limits. but i need to send another field to my handler

try {
  const options = {
    limits: { 
      fileSize: nconf.get("maxFileSize"), // 52428800
    }
  };
  const parts = await req.files(options);
  for await (const part of parts) {
    const filePath = `tmp/${part.filename}.${Date.now()}`; // Create an unique filename to prevent duplicates
    await pump(part.file, fs.createWriteStream(filePath));
    const fileSize = fs.statSync(filePath).size;

    if (part.file.truncated) {
      fs.unlinkSync(filePath);
      const rejectedFileDetails = {
        filename: part.filename,
        fileSize,
        reason: `File size too big`
      }
      response.rejected.push(rejectedFileDetails);
    } else {
      const uploadedFileDetails = {
        filename: part.filename,
        fileSize
      }
      response.uploaded.push(uploadedFileDetails);
    }
  }
} catch (error) {
  // Prevent too verbose errors to be sent to the client. Ideally the max file size will be limited by the frontend.
}

Is there something wrong?

*sorry for my bad english

Expected behavior

The file size I upload must match what I submitted from the form

Environment

• node version: v14.15.0
• fastify version: ^3.1.1
• fastify-multipart version: ^4.0.0
• os: Mac

[frapan]

I faced the same issue and proved that @simoneb was right.

As the documentation states, if you assign all fields to the body and don’t define an onFile handler, the file content will be accumulated in memory. Therefore, the stream is empty because it has already been read, causing a zero bytes file.

The only way to access the file content is through the toBuffer method or by defining a custom onFile handler.

I think the documentation might be more explicit about this. I’ll provide a PR asap to improve the documentation.

[simoneb]

Here’s a repro of the issue. Reading the stream won’t work, using toBuffer works.

The reason can be found in these lines of code. The file stream is read with toBuffer when the request is received, meaning that the stream won’t be readable anymore in the handler because it’s already been read.

I believe this is done because the request body needs to be processed in full before proceeding. I also verified that indeed the size limits are not validated eagerly but only after the request body is parsed.

Now this all looks like a bug but I guess it’s somewhat intended. As you can see in the linked code an alternative to using toBuffer would be to provide an onFile handler.

const fs = require("fs");
const pump = require("pump");

const server = require("fastify")();

server.register(require("."), {
  attachFieldsToBody: true,
});

const handler = async (req) => {
  const part = await req.body.file;
  const filePath = `uploads/${part.filename}`;

  // does not work
  await pump(part.file, fs.createWriteStream(filePath));

  // works
  // fs.writeFileSync(filePath, await part.toBuffer());

  return "ok";
};

server.post("/upload", handler);

server.listen(3000);