Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

How to finetune ELECTRA on glue?

See original GitHub issue

After pretraining my own electra model, I wanted to test it out in Glue using run_glue.py. However I got this:

ValueError: Unrecognized configuration class <class 'transformers.configuration_electra.ElectraConfig'> for this kind of AutoModel: AutoModelForSequenceClassification.
Model type should be one of DistilBertConfig, AlbertConfig, CamembertConfig, XLMRobertaConfig, BartConfig, RobertaConfig, BertConfig, XLNetConfig, FlaubertConfig, XLMConfig.

After taking a look at the source code, It seems like ElectraConfig isn’t available for sequence classification, is there a reason for that? Did anyone finetune electra on glue?

Issue Analytics

State:
Created 3 years ago
Comments:8 (6 by maintainers)

Top GitHub Comments

1reaction

elyesmanaicommented, May 22, 2020

awesome, it works perfectly, thank you very much!

0reactions

cktsangalcommented, Jul 14, 2020

This problem happened again, when I use ELECTRA on question-answering pipeline. My Transformers version is 2.11.0.

from transformers import pipeline, AutoTokenizer, AutoModelForQuestionAnswering

tokenizer = AutoTokenizer.from_pretrained(“ahotrod/electra_large_discriminator_squad2_512”)

model = AutoModelForQuestionAnswering.from_pretrained(“ahotrod/electra_large_discriminator_squad2_512”)

albert_qa = pipeline(‘question-answering’, model=model, tokenizer=tokenizer)