Error: write after end

https://gist.github.com/devTristan/5aa69c2822a0da3f5e51

If the stream fast-csv is piped to doesn’t always write immediately, I get a write after end error. This happens if it waits 100ms every 100 records, or every 10ms on every other record, or pretty much any other combination.

I’ve tried this with through and through2. I’m not familiar enough with streams to replicate this issue without using a helper module, so it’s possible that this bug is present in both through and through2 instead of fast-csv. However, I’m pretty sure this is a fast-csv thing, because I can’t replicate it using other csv readers.

Commenting out this line prevents this issue from happening, but it also causes a test to fail (specifically, the github issue test for #68).

Help me, @doug-martin, you’re my only hope.

[ifalldev]

Hello everybody!

I’m stuck in this issue. That is my stream pipeline:

const s3 = new AWS.S3({...})
const csvStream = csv({ headers: true }).
  on.('data', ...);
const stream = s3.getObject(params).createReadStream().pipe(csvStream)

And that is the error message: Error [ERR_STREAM_WRITE_AFTER_END]: write after end

I would appreciate any help, thanks!

[doug-martin]

@seamusabshere @devTristan Np thanks for using the library!