Error from read_feather (in the feather R library)

I’ve been getting the following error when using read_feather():

 *** caught segfault ***
address 0x7f9b75bab010, cause 'memory not mapped'

Traceback:
 1: .Call("feather_coldataFeather", PACKAGE = "feather", feather,     indexes)
 2: coldataFeather(x, i)
 3: `[.feather`(x, )
 4: x[]
 5: as_data_frame.feather(data)
 6: as_data_frame(data)
 7: read_feather(file.match)

My sessionInfo():

R version 3.2.1 (2015-06-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 14.04.3 LTS

locale:
 [1] LC_CTYPE=en_US.UTF-8          LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8           LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8       LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=en_US.UTF-8          LC_NAME=en_US.UTF-8
 [9] LC_ADDRESS=en_US.UTF-8        LC_TELEPHONE=en_US.UTF-8
[11] LC_MEASUREMENT=en_US.UTF-8    LC_IDENTIFICATION=en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
 [1] feather_0.0.1         metricsgraphics_0.8.5 dygraphs_0.5
 [4] RColorBrewer_1.1-2    htmlwidgets_0.5       htmltools_0.2.6
 [7] DT_0.1                shinydashboard_0.5.1  shiny_0.12.2
[10] lubridate_1.3.3       helpRs_0.1            magrittr_1.5
[13] ggplot2_2.1.0         RPostgres_0.1-1       data.table_1.9.6
[16] dplyr_0.4.3


loaded via a namespace (and not attached):
 [1] Rcpp_0.12.5          futile.logger_1.4.1  plyr_1.8.1
 [4] R.methodsS3_1.7.1    futile.options_1.0.0 R.utils_2.3.0
 [7] tools_3.2.1          digest_0.6.8         packrat_0.4.1-1
[10] tibble_1.0           lattice_0.20-31      memoise_0.2.1
[13] gtable_0.1.2         DBI_0.4-1            parallel_3.2.1
[16] rJava_0.9-8          mailR_0.4.1          stringr_1.0.0
[19] httr_1.1.0           grid_3.2.1           R6_2.1.2
[22] lambda.r_1.1.7       scales_0.4.0         urltools_1.4.0
[25] tldextract_1.0       assertthat_0.1       mime_0.4
[28] colorspace_1.2-4     xtable_1.7-4         httpuv_1.3.3
[31] stringi_1.0-1        munsell_0.4.2        chron_2.3-47
[34] zoo_1.7-11           R.oo_1.20.0

And the str(the.table_I_tried.to.save) of the object I’m writing:

Classes ‘data.table’ and 'data.frame':  1888384 obs. of  5 variables:
 $ uuid         : chr  "00105052-ff64-43ed-9405-2d4bf847ac40" "0001a6a0-2cf2-49ca-adc3-ff0cd87cf1aa" "00027a78-6774-48cc-a53f-6c985321d24f" "0000867e-666e-4c99-9bc0-1a153bd5e53e" ...
 $ status          : chr  "no data" "success" "no data" "success" ...
 $ update_date     : chr  "2016-02-15" "2016-02-15" "2016-02-15" "2016-02-15" ...
 $ table_name      : chr  "location" "location" "location" "location" ...
 $ last_update_date: chr  "2016-02-15" "2016-02-15" "2016-02-15" "2016-02-15" ...
 - attr(*, ".internal.selfref")=<externalptr>

I wish I could produce a reproducible example but I haven’t been able to come up with one, and unfortunately I can’t post the data I’m using. Fwiw, I tried writing a plain data.frame instead of a data.table but the same happened. Afaik there shouldn’t be anything funky in the values of the variables.

A couple more details I’ve noticed:

• This tends to happen with large data.frames
• If I first read the data.frame from an .rds file and then read the same object from a .feather file this error doesn’t happen (observed this while debugging).

[dtmngngnnj]

TL;DR: problem seems fixed!

I just installed the latest version of feather, did a quick test, and then a more thorough one, and can report that the problems is no longer. Thanks for the fix!

Detail:

BEFORE UPGRADE: sessionInfo() tells me version of feather is: [1] feather_0.0.1

Reinstall feather/R via devtools (did this on Thu Aug 11 2016, at 11h00 GMT)

devtools::install_github("wesm/feather/R")

AFTER UPGRADE: R sessionInfo now says: [1] feather_0.0.1.9000

A quick, short test, proved successfull, then I reran the longer test I described here:

http://data.munging.ninja/pubcoms/20160526_bug_feather_mem_not_mapped/

And it works fine! Great!

[statquant]

Hello, I am also getting this error, I am reading feather file in parallel and am getting random errors like Error in readFeather(filePath, gridTbl = dataGrid, columns) : task 223 failed - “SET_STRING_ELT() can only be applied to a ‘character vector’, not a ‘NULL’” Usually relaunching the exact same statement works