CSVParser - "OutOfMemoryError: GC overhead limit exceeded" with large file

my file has about 20,000,000 rows X 6 cols.

main code:

Table user = Table.read().csv("filename.csv");
out("Table name: " + user.name());

Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
	at java.lang.String.substring(String.java:1969)
	at com.opencsv.CSVParser$StringFragmentCopier.peekOutput(CSVParser.java:609)
	at com.opencsv.CSVParser$StringFragmentCopier.takeOutput(CSVParser.java:616)
	at com.opencsv.CSVParser.parseLine(CSVParser.java:421)
	at com.opencsv.CSVParser.parseLineMulti(CSVParser.java:309)
	at com.opencsv.CSVReader.readNext(CSVReader.java:338)
	at tech.tablesaw.io.csv.CsvReader.detectColumnTypes(CsvReader.java:430)
	at tech.tablesaw.io.csv.CsvReader.read(CsvReader.java:161)
	at tech.tablesaw.io.DataFrameReader.csv(DataFrameReader.java:59)
	at tech.tablesaw.io.DataFrameReader.csv(DataFrameReader.java:55)
	at tech.tablesaw.io.DataFrameReader.csv(DataFrameReader.java:35)
	at ReadCSV.main(ReadCSV.java:7)

when I set JVM: -Xms2g -Xmx2g -Xmn512M .

problem exits still.

ps. my laptop has 12g memory. my Java jdk version is 8.

[lwhite1]

it might be easier to get OpenCSV to fix the bug, if it is theirs, than to replace the CSV parser.

On Thu, Dec 7, 2017 at 4:42 PM, SALES [e-mail] wrote:

the Apache commons is well maintained.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jtablesaw/tablesaw/issues/208#issuecomment-350103094, or mute the thread https://github.com/notifications/unsubscribe-auth/ADRXgrxORjJR1s-4awPimQkMwNOoCFVPks5s-Fu3gaJpZM4Q3kWW .

[lwhite1]

fixed.