Ignore preprocessData option

Does preprocessData() do anything if opts.unwind and opts.flatten are not defined? As far as I can tell, it creates a single item array for each row and then concats them back together. This is very inefficient in the browser when working with 100k+ rows.

I read that streams are more efficient, but I didn’t see any documentation about using this in the browser. Not sure if streams make sense in the browser since everything is in memory anyways.

I was able to “fix” the issue by just overriding the preprocessData function in a very hacky way. I was hoping you’d provide an option to skip preprocessing.

const parser = new Parser({ fields }) as any;
parser.preprocessData = (x:Array<any>) => x;
const csv = parser.parse(data);

Exporting ~90 fields and ~90k rows went from taking 45+ seconds (or freezing) to a more reasonable 9 seconds.

I’m using v4.3.3 in Chromium v70. Let me know if you need any other data.

[randdusing]

Works well, thanks again.

[juanjoDiaz]

I’m working on a new API to use the streaming API in the browser easily. Just give me a couple days.