Unable to serialize/save TF2.0 Bert model

Environment info

https://colab.research.google.com/gist/Douboo/fccd6bcda2e098b10b1c7490f2d8bbf3/untitled3.ipynb#scrollTo=yW4t3uQxjhy1

• transformers version: 3.1.0
• Platform: colab
• Python version: 3.7.0
• Tensorflow version (GPU?): 2.3.0
• Using GPU in script?: yes

Who can help

@jplu

Information

Model I am using (Bert):

The problem arises when using:

• the official example scripts: (give details below)
• my own modified scripts: (give details below)

The tasks I am working on is:

• an official GLUE/SQUaD task: (give the name)
• my own task or dataset: (give details below)

To reproduce

# define model via functional api
def build_model(item_dim, num_layers, num_heads, max_len):
    config = BertConfig(hidden_size=item_dim, num_hidden_layers=num_layers,
                        num_attention_heads=num_heads, intermediate_size=item_dim*4,
                        max_position_embeddings=max_len)
    bert = TFBertMainLayer(config=config)
    inputs_embeds = Input(shape=(max_len, item_dim), dtype=tf.float32, name='inputs')
    inputs = {"inputs_embeds": inputs_embeds}
    # pre-training vectors to bert
    seq_emb = bert(inputs)[0]
    last_token_emb = seq_emb[:, -1, :]
    outputs = Dense(1, activation='sigmoid')(last_token_emb)
    model = Model(inputs=inputs, outputs=outputs)
    return model

model = build_model(item_dim, num_layers, num_heads, max_len)
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['acc'])

model.fit(X, y, 
          epochs=2, batch_size=128,
          verbose=2
         )

model.save('./my_model')

Errors with TypeError: ('Not JSON Serializable:', BertConfig { "attention_probs_dropout_prob": 0.1, "gradient_checkpointing": false, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 32, "initializer_range": 0.02, "intermediate_size": 128, "layer_norm_eps": 1e-12, "max_position_embeddings": 9, "model_type": "bert", "num_attention_heads": 1, "num_hidden_layers": 1, "pad_token_id": 0, "type_vocab_size": 2, "vocab_size": 30522 } )

Expected behavior

• Successfully saved keras model.

[jplu]

I can confirm the bug. As a workaround you can use model.save_weights instead. I cannot give you a specific date for the fix, but I will update the thread.

[Douboo]

I can confirm the bug. As a workaround you can use model.save_weights instead. I cannot give you a specific date for the fix, but I will update the thread.

ok, thanks!