[feature request] support for user-level metadata

Idea would be to allow a restricted metadata to be attached globally? per-column? of types that are serializable by feather. something like. The usecase I am thinking is to allow pandas to support complete round-tripping.

e.g.

In [3]: df = pd.DataFrame({'A' : ['foo','foo','bar'], 'B' : [1,2,3]})

In [4]: df
Out[4]: 
     A  B
0  foo  1
1  foo  2
2  bar  3

In [5]: feather.write_dataframe(df, 'foo.fth')

In [6]: feather.read_dataframe('foo.fth')
Out[6]: 
     A  B
0  foo  1
1  foo  2
2  bar  3

In [7]: feather.read_dataframe('foo.fth').equals(df)
Out[7]: True

In [8]: sdf = df.set_index('A')

In [9]: sdf
Out[9]: 
     B
A     
foo  1
foo  2
bar  3

In [10]: feather.write_dataframe(sdf, 'foo.fth')

In [11]: feather.read_dataframe('foo.fth')
Out[11]: 
   B
0  1
1  2
2  3

so on [11] loosing the ‘metadata’. But in a pandas wrapper if I could do:

feather.write_data_frame(sdf.reset_index(), 'foo.fth', {'index' : ['A']})

df, metadata = feather.read_frame('foo.fth', return_metadata=True)
df = df.set_index(metadata['index'])

Then I could round-trip exactly.

HDF5 allows support for arbitrary metadata on the group nodes (it just pickles this). But I think that would be out-of-scope here as its not cross-lang compat.

I think this would nicely solve the representation problem (everything still stored as table likes), but the metadata can offer an alternative interpretation w/o getting in the way of others.

[hadley]

@ScottPJones you’d be better off starting a new issue to discuss this

[wesm]

With Feather V2 coming in Arrow 0.17.0 we support all the metadata storage features that Arrow schemas have, which cover the use cases here I think