Error: Invalid: File is too small to be a well-formed file

When I imported a 423MB csv file (2,148,191 x 35) as a data table and saved it to feather, it created a ~626MB feather file which I could then read in to R easily, and when converting the tbl_df to a data.table it passed identical, let alone all.equal.

Trying the same with two larger files gives errors. A 4.7GB csv file (26,716,354 x 28) creates a ~6.2GB feather file, and a ~1.5GB csv (6,327,455 x 58) creates a 2.5GB feather file, but trying to read them in using the exact same path as used to write them returns: Error: Invalid: File is too small to be a well-formed file

I understand that means that some check value is not matching up, but what could it be? I performed the exact same procedure on the smaller file and it worked.

Thanks.

R version 3.3.0 Patched (2016-05-03 r70580)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252   
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] feather_0.0.1    data.table_1.9.6

loaded via a namespace (and not attached):
[1] assertthat_0.1 tools_3.3.0    tibble_1.0     Rcpp_0.12.5    chron_2.3-47 

[wesm]

Yep. It seems like enough people are hitting the 2 GB limit that we should fix this as soon as possible. I will try to get a patch up in the next day or two

[wesm]

It really depends on the data. For data with a compact text representation (e.g. lots of small integers, or floating point data without many decimal places), the binary representation may be more bytes per value on average.