Can't freeze pre-trained params

Thanks for releasing this repo, but I met a problem. I can.t freeze pre-trained params, I use the following code to frezze params, but it did’t work. `model_dir = “D:/ProgramData/Pre_Traines_Model_Of_Bert/chinese_L-12_H-768_A-12”

bert_params = bert.params_from_pretrained_ckpt(model_dir)
l_bert = bert.BertModelLayer.from_params(bert_params, name="bert")
l_bert.apply_adapter_freeze()
max_seq_len = 128
l_input_ids = keras.layers.Input(shape=(max_seq_len,), dtype='int32')
# l_token_type_ids = keras.layers.Input(shape=(max_seq_len,), dtype='int32')

# using the default token_type/segment id 0
output = l_bert(l_input_ids)  # output: [batch_size, max_seq_len, hidden_size]
model = keras.Model(inputs=l_input_ids, outputs=output)
model.build(input_shape=(None, max_seq_len))
bert.loader.load_stock_weights(l_bert, os.path.join(model_dir, "bert_model.ckpt"))
model.summary()`

And I get following result:

As you see, all params are trainable, Could u help me ASAP, THX! 😃

[ptamas88]

you may set it with l_bert.trainable = False. but it’s just a wild guess, I Don’t know what apply_adapter_freeze() function does.

[kpe]

@ck37 - apply_adapter_freeze() is used together with the adapter_size parameter, i.e. it will freeze all but the adapter layers for training; in the case you use plan bert (i.e. without adapter layers), the method will not have an effect.

To freeze your layers, you might use the standard keras mechanisms:

l_bert.trainable = False

just like @ptamas88 have suggested!