WER increases with increasing beam size, and with including LM

Hello,

I am running experiments with the WSJ corpus using PyTorch. I only use the seq2seq part of the code with --mtlalpha set to 0. During decoding, even without a language model, the CER, WER with greedy decoding is best and increasing the beam size only leads to increased errors.

Upon LM training on the WSJ corpus, the best model perplexity reached is:

2018-04-04 21:44:27,006 (lm_pytorch:248) INFO: iteration: 79400
2018-04-04 21:44:27,006 (lm_pytorch:249) INFO: training perplexity: 2.81606144442938
2018-04-04 21:45:16,637 (lm_pytorch:255) INFO: epoch: 24
2018-04-04 21:45:16,638 (lm_pytorch:256) INFO: validation perplexity: 4.354437210237004 

And decoding with including such a LM, worsens CER, WER results further.

I believe this might be due to some issue in the decoder or the beam search? Please let me know if you need any further details.

[shrutijpalaskar]

Hello, @sw005320 @ShigekiKarita @m-wiesner,

Here are the results for varying beam size with WSJ where error increases by increasing the beam size. The vertical axis shows character or word error rate and the horizontal axis shows beam size. My experiments parameters are:

backend = pytorch mtl alpha = 0 etype = blstmp bs = 48 ls = unigram 0.05 penalty = 0.0 max min ratio = 0.0, 0.0 ctc weight = 0 rnn_lm weights = 0.0

[atiklt1972]

Ok, Thanks for your immediate response.