Customizing both decoding and encoding of a type

I’m considering adopting pydantic in a project that serializes numpy arrays as base64-encoded gzipped strings. The current serialization solution allows registering type-specific hooks for encoding and decoding, so that after registering the hooks any field annotated as numpy.ndarray will automatically be handled correctly:

class Example(object):
    big_array: numpy.ndarray

ex = Example(big_array = numpy.arange(50))

serialized = serialize_to_dict(ex)
print(serialized)
# { 'big_array': 'H4sIAAEAAAAC/xXOyRHCUAwE0VQUgA7MArZjocg/Dfqf+5VG39eOdryTne68dz47186985BOpgsg\nhCDCCCSUYMIZ53MHZ5xxxhlnnHHGBRdcziAuuOCCCy644Iorrriez3DFFVdccX1+f1+8uIe+AAAA\n' }

deserialized = deserialize_dict(serialized, Example)
print(deserialized.big_array)
# array([0, ..., 49])

I noticed that Config has a json_encoders field that seems to allow this for encoding, but I haven’t seen a way to customize decoding (maybe I’m just too sleep-deprived). Is there a way to achieve the above behavior using pydantic?

[samuelcolvin]

The main implementation would be more or less that, but it would need to work recursively somehow so a field foobar: List[MyComplexThing] called __serialise__ on every member of the list.

I’m also concerned about what to do with standard types that might need simplifying for output.

Perhaps we should do something like #317, e.g.:

• we add a simplify kwarg to dict()
• simplify=True causes pydantic_encoder to be called recursively on the the dict, would need modifying to look more like jsonable_encoder
• pydantic_encoder looks for __serialise__ and calls it if it exists, thus model.json() would work with __serialise__ without the slow down of simplify=True.

What do you think?

Looking at that, I’m not sure implementing this will as simple as initially though, especially given that performance is important so we’ll probably need a micro-benchmark. Feel free to start a PR, otherwise I’ll work on it in a couple of weeks.

[stur86]

Has this gone ahead? I’m interested as well, seems like a very useful feature that makes custom types come onto their own.