Multiple Inputs

I was wondering if it is possible to use this library to build a model, which can have multiple inputs.

Like:

inputs1 = ...
inputs2 = ...
inputs1_embedded = bert_embedding()(inputs1)
inputs2_embedded = bert_embedding()(inputs2)
model_out1 = model()(inputs1_embedded)
model_out2 = model()(inputs2_embedded)
....

[CyberZHG]

For the training parameter in load_trained_model_from_checkpoint function, does it simply means no masked language layer after encoders output if I set the parameter to True ?

No masked language layer if it is False.

When I was reading your Chinese README, it says “如果不想再训练嵌入层，可以传入trainable=['Encoder']来只对编码层进行调整”. Does this means if set trainable to [encoder] I am not fine tuning the bert’s embedding layers?

Yes.

[CyberZHG]

Model is callable just like Layers:

import os
import keras
from keras_bert import load_trained_model_from_checkpoint

inputs_1 = [
    keras.layers.Input(shape=(16,), name='Input-Token-1'),
    keras.layers.Input(shape=(16,), name='Input-Segment-1'),
]
inputs_2 = [
    keras.layers.Input(shape=(16,), name='Input-Token-2'),
    keras.layers.Input(shape=(16,), name='Input-Segment-2'),
]

current_path = os.path.dirname(os.path.abspath(__file__))
config_path = os.path.join(current_path, 'test_checkpoint', 'config.json')
model_path = os.path.join(current_path, 'test_checkpoint', 'model.ckpt-20')
model = load_trained_model_from_checkpoint(
    config_path,
    model_path,
    training=False,
    trainable=['Encoder'],
)

outputs_1 = model(inputs_1)
outputs_2 = model(inputs_2)
model = keras.models.Model(inputs=inputs_1 + inputs_2, outputs=[outputs_1, outputs_2])
model.summary()