Loading fine tuned BertForMaskedLM

Hi,

I tried to fine tune BertForMaskedLM and it works. But i’m facing issues when I try to load the fine tuned model. Here is the code I used to load the model :

model_state_dict = torch.load("./finetunedmodel/pytorch_model.bin",  map_location='cpu')
model_fine = BertForMaskedLM.from_pretrained(pretrained_model_name='bert-base-multilingual-cased', state_dict=model_state_dict, cache_dir='./data')

The error I’m facing is : init() got an unexpected keyword argument ‘state_dict’

Does someone already faced this issue ?

Thanks

Edit : I trained my model on gpu and try to use it on cpu. When I use it on gpu it works !

[mraj96]

yeah yes. if you trained model in GPU, can’t be loaded. we will change map_location=“CPU” in modeling.py line 511. https://github.com/huggingface/pytorch-pretrained-BERT/blob/0a9d7c7edb20a3e82cfbb4b72515575543784823/pytorch_pretrained_bert/modeling.py#L511

state_dict = torch.load(weights_path, map_location="CPU" )

now it will load our finetuned model. this is only for inference running on CPU.

[psureshmagadi17]

@tgriseau - I want to fine-tune bert on MaskedLM using domain-specific text. could you please provide an example of how you fine-tuned or provide some details about what kind of inputs need to be passed? will I be using the true sentence as the output for fine-tuning?