tensorflow2_gpt2 Slow speed

❓ The speed of pytorch_gpt2 based on transformers is 4-5 times faster than that of tensorflow_gpt2, which is unreasonable. Where is my problem

pytorch_gpt2

tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = GPT2LMHeadModel.from_pretrained('gpt2')

model.to(dev)
raw_text_2 = "And if so be ye can descrive what ye bear,"

inputs = tokenizer.encode(raw_text_2, add_special_tokens=False, return_tensors="pt")
inputs = inputs.to(dev)
generated = model.generate(inputs, top_k=0, max_length=512)

tensorflow2_gpt2

tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = TFGPT2LMHeadModel.from_pretrained('gpt2')

raw_text = "And if so be ye can descrive what ye bear,"

tokens = tokenizer.encode(raw_text,  return_tensors="tf")
output_ids = model.generate(tokens, top_k = 0, max_length=512)

[only-yao]

With tf. Varibable model for packaging

    @tf.function(experimental_relax_shapes=True)
    def model_static(self, model_inputs):
        outputs = self(**model_inputs)
        return outputs

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.