How to set data type to whole Table.

Hello tablesaw team. I’ve question regarding setup of data type for the Table if I don’t know the amount of colums. I have 30_000x2000 feature csv file with 0.0 and some other amount of Double numbers. If I call csv parsing via:

CsvReadOptions options = CsvReadOptions.builder(csv)
                    .header(false)
                    .maxNumberOfColumns(50_000).build();

Table t = Table.read().csv(options);

I got Number format exception, as all 0.0 number are treated as Short 0. So when reader gets to real numbers like 13.5 if throws NFE.

But if I add sample(false) to reader options if takes about 2:40 to parse such file.

How can I setup data type for whole Table, as far as I can see only by setting columnType in parser option, but it’s won’t work as I don’t know a number of columns on csv file?

P.S. I used com.univocity.parsers.csv.CsvParser separately to read the same file so it takes 2:20 for parser.parsAll and 1:20 for parsing file by row.

[Ebalaitung]

Yes, exactly!

[lwhite1]

if you’re reading a CSV file, there is an option in CsvReadOptions that lets you specify the types to be used when you read the file. If you just provide one type (e.g. STRING), they would all be the same. IDK if this is implemented for Excel, however.

On Tue, Jan 25, 2022 at 9:28 AM Wesley @.***> wrote:

I know this is closed, but I’m going to plug for this functionality anyway. It doesn’t seem “too specialized” at all.

The suggestion to “read a table then copy and paste its types” is far from programmatic and becomes impossible to automate. If we were all just working in Jupyter notebooks, I’d not be complaining but I’m trying to develop software.

— Reply to this email directly, view it on GitHub https://github.com/jtablesaw/tablesaw/issues/415#issuecomment-1021237092, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA2FPAVCJDTZD2W7ORSQVMLUX2XPBANCNFSM4GFDZ2NA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>