module 'tokenization' has no attribute 'FullTokenizer'

I’m importing tokenization, have installed via pip, and cannot instantiate the tokenizer. I’m using the following code below and continue to get an error message of “module ‘tokenization’ has no attribute ‘FullTokenizer’”.

Anyone have a sense as to why?

tokenizer = tokenization.FullTokenizer(vocab_file=vocab_file, do_lower_case=do_lower_case)

[nicsuzor]

If you installed from pip (I’m presuming pip install bert-tensorflow), try: from bert import tokenization

[celsofranssa]

Maybe it can help someone using Tensorflow 2 and bert-for-tf2. There was a little change to create an instance of FullTokenizer:

from bert import bert_tokenization

...

tokenizer = bert_tokenization.FullTokenizer(
        vocab_file="resources/model/bert/bert_vocab.txt", 
        do_lower_case=True
    )