Error using inputs_embeds argument in TFXLNetModel

While using the TFXLNetModel: xlnet = TFXLNetModel.from_pretrained('xlnet-base-cased') according to the docs, input_ids and inputs_embeds can be optionally used. However, when I tried using:

xlnet(inputs_embeds=embeddings, attention_mask=attn_masks)[0] it throws: ValueError: The first argument to Layer.call must always be passed.

which I thought is an issue with the inputs argument which must be a positional one: xlnet(inputs=None, inputs_embeds=embeddings, attention_mask=attn_masks)[0] using this gave me: RuntimeError: Attempting to capture an EagerTensor without building a function.

And finally passing both inputs and inputs_embeds gave: ValueError: You cannot specify both input_ids and inputs_embeds at the same time

Can someone suggest a workaround on this? P.S. the embeddings variable is the last_hidden_state from another bert which matches the config for the inputs_embeds shape. Note that input_ids parameter won’t count as it gave the same error if I didn’t use the inputs argument.

[AndyTheFactory]

While using the TFXLNetModel: xlnet = TFXLNetModel.from_pretrained('xlnet-base-cased') according to the docs, input_ids and inputs_embeds can be optionally used. However, when I tried using:

xlnet(inputs_embeds=embeddings, attention_mask=attn_masks)[0] it throws: ValueError: The first argument to Layer.call must always be passed.

try something like this: xlnet({‘attention_mask’:attention_mask, ‘token_type_ids’:token_type_ids},inputs_embeds=embeddings, training=training)

worked for me when getting the same error for a bert model

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.