Converting fairseq NMT to transformers misses model weight

Hi there, question about fairseq NMT model (FSMT) conversion.

I tried to convert my own fairseq-nmt model (transformer_wmt_en_de) based on this conversion script. However, decoder.embed_out weight is missing after converting fairseq model to transformers FSMT model. This parameter exists when not specifing --share-all-embeddings or --share-decoder-input-output-embed, while official fairseq wmt models do not have decoder.embed_out weight because specifying --share-all-embedding. https://github.com/pytorch/fairseq/issues/2537

Are there any solution or tips to converting own fairseq model?

[stas00]

Thank you for clarifying that your original issue has been resolved. Please feel free to close this issue when you feel it’s working for you.

Based on your comments, I’m concerned about 2 things:

1. your different dictionaries - a model has to come with the exact dict it was trained on, after conversion too. So it sounds that something isn’t right there. If you’re not sure what’s happening perhaps try to clarify how it came to be that your fairseq model has a different vocab size.
2. perhaps that output_projection layer is getting in the way of your model if it was trained without it. You could try to hunt down the few lines where it’s used in the code and and bypass it and test whether your translation works then. If you’re comfortable editing the source code that is.

[NielsRogge]

Pinging @stas00 here