Unable to use fill-mask pipeline on gpt-neo model

Environment info

• transformers version: 4.5.0
• Platform: googleColab
• Python version:3.7

Models: GPT neo

Code :

#Import Hugging Face's Transformers
from transformers import pipeline
generator = pipeline('fill-mask', model='EleutherAI/gpt-neo-1.3B')

Error:

Can someone help me know what could the reason be for not able to use the fill-mask on gpt-neo model?

[NielsRogge]

Isn’t the text generation specific to generating new text with a given prompt? => well, normally it is meant to generate new text given a prompt indeed. But as models like GPT-3 and GPT-neo are so powerful and are trained on a lot of data, they are capable of performing what the authors of GPT-3 call “in-context learning”: this means that the model knows what to do just based on a given prompt. See the GPT-3 paper for more info.

I’ve just tried it with GPT-3 and it works. However, GPT-neo doesn’t seem as powerful. This is logical since GPT-3 has 175 billion parameters, whereas GPT-neo only has 1.3 billion (there’s also a 2.7 billion variant available).

Maybe you can try by giving more examples in the prompt. Sometimes it seems to work:

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.