Is a way to check the file size first then decide if the file is saving into the disk?

Hi,

I was assuming file.on(data, function () {...}) can handle this, for example:

var busParams = {headers: req.headers, limits : {fileSize:10000, files:1}};
var busboy = new Busboy(busParams);
var filePath = path.join(path.normalize(__dirname + '/..'), 'public', 'images');

busboy.on('file', function (fieldname, file, filename, encoding, mimetype) {
        file.on('data', function () {
            if (data.size > busParams.limits.fileSize) {
                return console.log('FILE TOO BIG');
            } else {
                var saveTo = filePath + '/' + Date.now() + filename;
                var fStream = fs.createWriteStream(saveTo);
                file.pipe(fStream);
                file.on('end', function () {
                    console.log('FILE UPLOADED')
                });
            }
        });
});

However, the uploaded file is empty. And I just realize that if file.pipe(fStream) is running inside of any event listener, then it gives an empty file which is reasonable but making the 'data' event not so useful. And the file have to save into the server memory first then being drained (by fs.unlink(saveTo) inside file.on('limit', function () {})), which is not so efficient.

So I just wonder if there is a way to check the file size first then decide if the file is saving into the disk?

[mscdex]

The reason why I see it is not so efficient is because, let’s say I set the size limits to 1GB; however, a client somehow send me 1.01GB of data, which will be drained once it exceed the limit, but it ate my server’s memory. And if the uploading seed from the client is slow, it just not good for the user’s experiences also.

Ultimately there is no way around this, no matter how you accept files from a user. This is not an issue limited to multipart forms. For example, if someone sends an HTTP PUT request with Transfer-Encoding: chunked, you won’t know you’ve reached the end until the client tells you. Meanwhile you have to do something with the data because you don’t know when the request stream will end (even malicious clients could lie about file length fields you may explicitly send on the front end). So you either Buffer it in memory (not a good idea generally) or you save it to disk somewhere or stream it to some other network resource (e.g. S3). No matter what though, resources somewhere will be used at least temporarily until the request stream has finished.

[leoyli]

Thank you very much for those details! I learn a lot from it!