How to load a pretrained TF model using AutoModel?

Run the following code:

import tensorflow as tf
from transformers import AutoModel, TFBertModel

auto_model = AutoModel.from_pretrained("bert-base-uncased")
tfbert_model = TFBertModel.from_pretrained("bert-base-uncased")

print(auto_model.__class__)
print(tfbert_model.__class__)

Then the output is:

<class 'transformers.modeling_bert.BertModel'>
<class 'transformers.modeling_tf_bert.TFBertModel'>

It seems that AutoModel defaultly loads the pretrained PyTorch models, but how can I use it to load a pretrained TF model?

[juanmanuelvc]

I can’t able to load model for model = TFAutoModel.from_pretrained(“emilyalsentzer/Bio_ClinicalBERT”) and TFAutoModel .from_pretrained(‘microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract-fulltext’) anyone ?

HI @blmali, I had the same issue when trying to load “emilyalsentzer/Bio_Discharge_Summary_BERT”. I solved it by passing from_pt argument as True: model = TFAutoModel.from_pretrained("emilyalsentzer/Bio_Discharge_Summary_BERT", from_pt=True).

I hope this helps.

[julien-c]

Hi @erikchwang, you should use TFAutoModel instead