Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Only first sheet is returning when working with Huge data sets

See original GitHub issue

While working or bigger data sets( tested with 2000 cols X 1000 rows) with multiple sheets. In that case, after certain MB, (in my case 40MB), sheet_to_row_object_array is only returning first sheet.

Is it a limitation? Are there any alternatives to get all the sheets regardless of files size?

Issue Analytics

State:
Created 7 years ago
Comments:15 (7 by maintainers)

Top GitHub Comments

1reaction

SheetJSDevcommented, Feb 26, 2017

@mtharrison scratch that.

The nodejs string limit is 16 bytes shy of 256MB: https://github.com/nodejs/node/issues/3175 . I thought the previous write test case would cover it, but the underlying xml files are 60M.

I’m going to resolve this in the next version by trying to catch that particular error and give more informative output.

Can you do one last thing: try saving that file as XLSB in Excel and seeing if you hit the same problem? The XLSB parser keeps data as buffers so it theoretically should not trigger a 256MB string conversion

1reaction

mtharrisoncommented, Feb 26, 2017

Thanks @SheetJSDev for the super-quick response. I tried with the debug option and the error I get is:

Error: Uncaught error: "toString()" failed
    at Buffer.toString (buffer.js:495:11)
    at getdatastr (/REDACTED/node_modules/xlsx/xlsx.js:1318:72)
    at getdata (/REDACTED/node_modules/xlsx/xlsx.js:1336:95)
    at getzipdata (/REDACTED/node_modules/xlsx/xlsx.js:1352:19)
    at safe_parse_ws (/REDACTED/node_modules/xlsx/xlsx.js:12165:26)
    at parse_zip (/REDACTED/node_modules/xlsx/xlsx.js:12268:3)
    at read_zip (/REDACTED/node_modules/xlsx/xlsx.js:12408:9)
    at Object.readSync [as read] (/REDACTED/node_modules/xlsx/xlsx.js:12420:21)

Looks like I’m probably hitting V8’s max string size. I logged the buffer size out before toString() is called on it and it’s 356mb.