m2m: generate OOMs on v100

I ran the downloads and the documented “generate on a v100” command:

fairseq-generate \
    data_bin \
    --batch-size 1 \
    --path 12b_last_checkpoint.pt \
    --fixed-dictionary model_dict.128k.txt \
    -s de -t fr \
    --remove-bpe 'sentencepiece' \
    --beam 5 \
    --task translation_multi_simple_epoch \
    --lang-pairs language_pairs.txt \
    --decoder-langtok --encoder-langtok src \
    --gen-subset test \
    --fp16 \
    --dataset-impl mmap \
    --distributed-world-size 1 --distributed-no-spawn \
    --pipeline-model-parallel \
    --pipeline-chunks 1 \
    --pipeline-encoder-balance '[26]' \
    --pipeline-encoder-devices '[0]' \
    --pipeline-decoder-balance '[1,24,1]' \
    --pipeline-decoder-devices '[0,1,0]' > gen_out

on a V100 w torch 1.5 and I got OOM.

fairscale==0.0.3
fairseq # pip install -e . from source at 9b0611e6
torch==1.5.1+cu101

Questions

1. Has this command worked for others?
2. Does anyone have a working generate command that takes advantage of multiple gpus?

cc: @shruti-bh

Thanks in advance!

[shruti-bh]

I will try to get these models and commands in by end of this week or early next week!

[sshleifer]

that worked! (on 6debe291) Thanks!