Memory-related error in R 3.2.4

I created a large data frame in Python and saved it using write_feather. When I try loading it in R, R crashes with the following error:

> results <- feather::read_feather('results.feather')
R(7404,0x7fff78524000) malloc: *** error for object 0x10c4d3630: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug

It makes no difference whether I use :: or load the package first.

However, when I reinstall feather and try to load results in the same R session, it works fine:

> install.packages('feather')
trying URL 'https://cloud.r-project.org/bin/macosx/mavericks/contrib/3.2/feather_0.0.1.tgz'
Content type 'application/x-gzip' length 649676 bytes (634 KB)
==================================================
downloaded 634 KB


The downloaded binary packages are in
    /var/folders/z6/mcdmp2d926bd5rh1825k1hnh0000gp/T//RtmpDBxipP/downloaded_packages
> results <- feather::read_feather('results.feather')
There were 15 warnings (use warnings() to see them)
> dim(results)
[1] 526595     50
> results <- feather::read_feather('results.feather')
There were 15 warnings (use warnings() to see them)
> dim(results)
[1] 526595     50
> warnings()
Warning messages:
1: Coercing int64 to double
2: Coercing int64 to double
3: Coercing int64 to double
4: Coercing int64 to double
5: Coercing int64 to double
6: Coercing int64 to double
7: Coercing int64 to double
8: Coercing int64 to double
9: Coercing int64 to double
10: Coercing int64 to double
11: Coercing int64 to double
12: Coercing int64 to double
13: Coercing int64 to double
14: Coercing int64 to double
15: Coercing int64 to double

And when I load a smaller data set first, I am able to load the large data set afterwards without a problem:

> metadata <- feather::read_feather('metadata.feather')
Warning messages:
1: Coercing int64 to double
2: Coercing int64 to double
3: Coercing int64 to double
4: Coercing int64 to double
5: Coercing int64 to double
6: Coercing int64 to double
7: Coercing int64 to double
8: Coercing int64 to double
9: Coercing int64 to double
10: Coercing int64 to double
> dim(metadata)
[1] 2885   60
> results <- feather::read_feather('results.feather')
There were 15 warnings (use warnings() to see them)
> dim(results)
[1] 526595     50
> warnings()
Warning messages:
1: Coercing int64 to double
2: Coercing int64 to double
3: Coercing int64 to double
4: Coercing int64 to double
5: Coercing int64 to double
6: Coercing int64 to double
7: Coercing int64 to double
8: Coercing int64 to double
9: Coercing int64 to double
10: Coercing int64 to double
11: Coercing int64 to double
12: Coercing int64 to double
13: Coercing int64 to double
14: Coercing int64 to double
15: Coercing int64 to double

The IRKernel in the Jupyter Notebook crashes hard when trying to load results.feather. When trying to load metadata.feather, the IRKernel sometimes returns the following error:

Error in coldataFeather(x, i): REAL() can only be applied to a 'numeric', not a 'NULL'

but I wasn’t able to replicate the error in general, and I’m currently using both data sets in the IRKernel without an apparent problem.

I get no errors or crashes in Python operating on this data.

System and package info:

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.11.5
BuildVersion:   15F34

$ R --version
R version 3.2.4 (2016-03-10) -- "Very Secure Dishes"
Copyright (C) 2016 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin13.4.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
http://www.gnu.org/licenses/.

$ Rscript -e "packageVersion('feather')"
[1] ‘0.0.1’

$ pip show feather-format

---
Metadata-Version: 1.1
Name: feather-format
Version: 0.2.0
Summary: Python interface to the Apache Arrow-based Feather File Format
Home-page: http://github.com/wesm/feather
Author: Wes McKinney
Author-email: [e-mail]
License: Apache License, Version 2.0
Location: /usr/local/var/pyenv/versions/rocketrip-datascience/lib/python3.5/site-packages
Requires: cython
Classifiers:
  Development Status :: 3 - Alpha
  Environment :: Console
  Programming Language :: Python
  Programming Language :: Python :: 2
  Programming Language :: Python :: 3
  Programming Language :: Python :: 2.7
  Programming Language :: Python :: 3.4
  Programming Language :: Python :: 3.5
  Programming Language :: Cython

[hadley]

It’s unlikely to be directly a memory problem but instead a bug somewhere in the R code. Is there anyway you could share the file? (Or even better code that generates the file).

[hadley]

Have you tried the development version of feather? It’s likely to fix this problem.