Write nested Pocos as Parquet

I’d like to get the bytes from generated parquet Poco-structure using the ChoParquetWriter

byte[] bytes = ChoParquetWriter.SerializeAll<MyData>(data);

The poco structure (IEnumerable<MyData> data as serialized json)

[{
	"Health": {
		"Id": 99,
		"Status": false
	},
	"Safety": {
		"Id": 3,
		"Fire": 1
	},
	"Climate": [{
		"Id": 0,
		"State": 2
	}]
}]

MyData.cs

public class MyData
{
    public Health Health { get; set; }
    public Safety Safety { get; set; }
    public List<Climate> Climate { get; set; }
}

(MyData is actually even more nested but follows the same pattern)

but this gives an error: Parquet: CLR type ‘<redacted>.Climate’ is not supported, please specify one of 'System.DateTimeOffset, System.DateTime, Parquet.File.Values.Primitives.Interval, System.Decimal, System.Boolean, System.Byte, System.SByte, System.Int16, System.UInt16, System.Int32, System.Int64, System.Numerics.BigInteger, System.Single, System.Double, System.String, System.Byte[], , , ’ or use an alternative constructor.

Excluding the Climate property and then it all works fine, what am I missing?

[bc3tech]

Because Parquet file doesn’t support nested data format. You will need to flatten it before storing them

Confused by this when docs seem to say nested columns are supported…

[Cinchoo]

Because Parquet file doesn’t support nested data format. You will need to flatten it before storing them

Here is one way to handle it

using (var r = ChoJSONReader<MyData>.LoadText(json)
    .UseJsonSerialization())
{
    using (var w = new ChoParquetWriter("MyData.parquet")
        )
    {
        w.Write(r.Select(rec1 => rec1.ToDictionary().Flatten().ToDictionary()));
    }
}