How to generate data using beam search from a custom gpt2 model?

❓ Questions & Help

Details

I have a custom model with classification and an LM head. `

    self.config = AutoConfig.from_pretrained("gpt2", num_labels=3)
    self.base_model = AutoModel.from_pretrained("gpt2", config=self.config)
    self.classifier = nn.Sequential(
        nn.Linear(self.config.hidden_size, self.config.num_labels),
    )

    self.lm_head = nn.Linear(self.base_model.config.n_embd, self.base_model.config.vocab_size, bias=False)`

I want to generate the sentences using this model (given the initial prefix) via beam search. How can I achieve that?

I know that LM with double head exists but it’s not fit for my usecase

[LysandreJik]

I would recommend you check the source code for the generate method and see how the beam search is implemented. It is not trivial, however.

Maybe @sshleifer and @patrickvonplaten have better tips on how to best do this.

[talha1503]

@nrjvarshney Did you find any suitable way to use generate function for a custom model? I am facing a similar issue with a model of mine, and would be really grateful if you could let me know how to solve the issue.