It's impossible to parse CSV file with more than 512 columns
See original GitHub issueI try to parse feature index file into Table (it’s about 2000x36000 matrix).
I try to use:
Table t = Table.read().csv(csv);
but got error from com.univocity.parsers.csv.CsvParser (log is attached 512-ERROR.txt) that there is more than 512 columns.
I’ve checked sources and found out that you create CsvParser with default settings, and there is no option to change some how this settings from your API.
I think there should be some way to change com.univocity.parsers.csv.CsvParserSettings from Table.read() place, or at least change columns size limit.
Issue Analytics
- State:
- Created 5 years ago
- Comments:7 (5 by maintainers)
Top Results From Across the Web
Unable to import csv file · Issue #3404 · sequelpro ... - GitHub
The CSV was read as containing more than 512 columns, more than the maximum columns permitted for speed reasons by Sequel Pro.
Read more >How to import csv files with massive column count into Apache ...
I'm running into a problem importing multiple small csv files with over 250000 columns of float64 into Apache Spark 2.0 running as a...
Read more >Easy way to import messy CSV files into Excel The ... - YouTube
If you constantly need to deal with ugly CSV files that do not split the data into columns when you open them in...
Read more >5 most common parsing errors in CSV files (and ... - Medium
#2 — Text field with an unescaped delimiter ... If the column separator appears unescaped in a text field, this will cause the...
Read more >CSV not parsed into columns despite Get Data delimiter set
Yes, I see. On preview stage Excel make some guesses how to transform your data, not always correct ones. It's always better to...
Read more >Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start FreeTop Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
Top GitHub Comments
I wonder if we should just hardcode
parserSettings.setMaxColumns(Integer.MAX_VALUE)
. I don’t really understand why there would be a column limitI’ve create PR for default value of 10_000