Corrupted files

Support plan

• which support plan is this issue covered by? (e.g. Community, Sponsor, or Enterprise): Community
• is this issue currently blocking your project? (yes/no): yes
• is this issue affecting a production system? (yes/no): no

Context

• node version: v14.15.4
• module (formidable) version: ^2.0.0-canary.20200504.1
• environment (e.g. node, browser, native, OS): Node, Postman
• used with (i.e. popular names of modules): Express
• any other relevant information:

What are you trying to achieve or the steps to reproduce?

I’m trying to upload a file within the maximum file size and despite Formidable giving me matching bytes expected and bytes received the file is stored corrupted.

Here’s my code:

router.js

...
router.get('/api/upload', (req, res) => fileHandler.uploadFile(req, res))
...

fileHandler.js

function uploadFile(req, res) {
    form.once('error', ({ message }) => {
        res.status(500).json({ message });
        return;
    })

    form.parse(req, (err, fields, files) => {
        console.log(fields);
        return res.json(files);
    })
}

The form is imported from another file called formDataHandler.js, which is where I setup Formidable. formDataHandler.js

const formidable = require('formidable');

const maxFileSize = 5 * 1024 * 1024;
const uploadDir = "./upload";
...
const options = {
    maxFileSize,
    uploadDir,
    multiples: true,
    keepExtensions: true,
    allowEmptyFiles: false,
    hash: "sha256"
}
...
let form = formidable(options);
...
form.on('fileBegin', (filename, file) => { handleFile(filename, file, uploadDir) })
...
function handleFile(filename, file, uploadDir) {
    const fileType = file.type.split("/").pop().toLowerCase();
    const dateTime = new Date().toISOString().replace(/:/g, "-");
    const id = Math.random() * 100;
    const renamedFile = `${dateTime}.${id}.${fileType}`;
...
    file.path = `${uploadDir}/${renamedFile}`;
    file.name = renamedFile;
...
}

Tried uploading it using examples/with-express.js and upload-multiple-files.js. Still got corrupted files here and here.

Rolling back to 1.2.2 solved the problem, but I can’t send a request after the first one because Express keeps returning “Cannot set headers after they are sent to the client”. I tried updating again: the file doesn’t get corrupted for the first couple of requests, then it’s downhill again…

Is there some variable cleaning I’m missing? I didn’t see anything related to this in the docs. I even tried cleaning the form variable in the end event. It makes no difference.

What was the result you got?

In the resulting JSON I get a different size, like 1798715, and a new hash every single time. Visually speaking, this is what I get. If I keep trying soon 0 bytes are uploaded even though, according to the progress event, every byte expected was received successfully.

What result did you expect?

The original file uploaded, expect for the original file name.

[alvarengi]

Sorry taking so long, guys.

I fixed the corrupted files in every example by downgrading to 1.2.2 then upgrading to 2.0.0-canary.20200504.1 again.

@tunnckoCore well, I tried copying everything from formDataHandler to uploadFile and it worked. Yay! Didn’t know splitting code like that would lead to problems like that. Lesson learnt!

Thanks for everything, @tunnckoCore and @GrosSacASac. Keep up the good work.

[tunnckoCore]

formidable(options); needs to be used on every request. Is that the case ? Otherwise I see nothing wrong.

I thought about that too.

@ialvarenga try initializing formidable() in the uploadFile. It’s the usual case that every middleware is doing, whether it is multer, koa-better-body or similar - we initialize Formidable, inside the middleware.

Other than that… I literally have no ideas for now.