BUG: can't grid search optimizers

It turns out that Keras optimizers are not picklable because they use lambdas inside functions 🤦

This can be fixed by using tf.keras.optimizers.serialize and tf.keras.optimizers.deserialize.

I think that maybe we need to list out all of the keras object types Model, optimizers, callbacks, etc. and use copyreg.pickle to register how to pickle each one using their custom keras serialize/deserialize methods.

[adriangb]

So the RFC was approved on the Keras end, but it did not cover the specific use case here, which includes optimizers, losses, metrics and callbacks. The pushback on these was mainly surrounding optimizers, given that there is no singular “official” API for serializing optimizers with their weights. Instead, we would have to call tf.keras.optimizers.Optimizer.get_weights/tf.keras.optimizers.Optimizer.set_weights.

We may not be able to upstream this since there is no clear use case in Keras for serializing optimizers, callbacks, etc. outside of a Model.

[adriangb]

Resolved by #128