Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Can not successfully run the evaluation script of speech_text_joint_to_text pre-training code

See original GitHub issue

🐛 Bug

The evaluation code for the Librispeech ASR Pre-training in https://github.com/facebookresearch/fairseq/blob/main/examples/speech_text_joint_to_text/docs/pre-training.md seems not to be well tested

To Reproduce

Command: python ./fairseq_cli/generate.py \ $S2T_DATA_PATH \ --task speech_text_joint_to_text \ --max-tokens 800000 \ --max-source-positions 800000 \ --nbest 1 \ --results-path $SAVE_PATH \ --batch-size 512 \ --path $FINAL_MODEL \ --gen-subset $SUBSET \ --config-yaml config.yaml \ --scoring wer \ --beam 10 --lenpen 1.0 --user-dir examples/speech_text_joint_to_text

The evaluation command for Librispeech ASR Pre-training has an error. I think we should add “–user-dir” before “examples/speech_text_joint_to_text”
After fixing the above issue, I directly evaluate the Fine-tuned model provided in https://github.com/facebookresearch/fairseq/blob/main/examples/speech_text_joint_to_text/docs/pre-training.md. I got another error: “OSError: Model file not found: /fsx/yuntang/2021/joint_pretraining_ASR/pretrain03/checkpoints/expt10_960h.wd0.01.config.neuu.lr_0.001.elr_1e-06.mu800.0k.uf6.bs200.msp1024.mtp1024.mtt3072.mspch600.0k.mass750.0k.miss64.0k.mst750.0k.dsb3.mask0.3.mr0.1.ssmp0.3.sump0.7.mwd.noscale.gelu.default.all.nb.lpos.dp0.1.bart.ngpu16/checkpoint6.pt”
Then, I directly evaluate fine-tuned model trained by myself. Then I get the following error: Traceback (most recent call last): File “./fairseq_cli/generate.py”, line 417, in <module> cli_main() File “./fairseq_cli/generate.py”, line 413, in cli_main main(args) File “./fairseq_cli/generate.py”, line 48, in main return _main(cfg, h) File “./fairseq_cli/generate.py”, line 201, in _main hypos = task.inference_step( File “/tmp/code/examples/speech_text_joint_to_text/tasks/speech_text_joint.py”, line 216, in inference_step return generator.generate( File “/opt/conda/lib/python3.8/site-packages/torch/autograd/grad_mode.py”, line 28, in decorate_context return func(*args, **kwargs) File “/tmp/code/fairseq/sequence_generator.py”, line 191, in generate return self._generate(sample, **kwargs) File “/tmp/code/fairseq/sequence_generator.py”, line 266, in _generate encoder_outs = self.model.reorder_encoder_out(encoder_outs, new_order) File “/tmp/code/fairseq/sequence_generator.py”, line 873, in reorder_encoder_out model.encoder.reorder_encoder_out(encoder_outs[i], new_order) File “/tmp/code/examples/speech_text_joint_to_text/models/s2t_dualinputtransformer.py”, line 377, in reorder_encoder_out return self.spch_encoder.reorder_encoder_out(encoder_out, new_order) File “/tmp/code/fairseq/models/speech_to_text/s2t_wav_transformer.py”, line 485, in reorder_encoder_out return self.speech_encoder.reorder_encoder_out(encoder_out, new_order) File “/tmp/code/fairseq/models/speech_to_text/s2t_wav_transformer.py”, line 381, in reorder_encoder_out if len(encoder_out[“encoder_out”]) == 0 TypeError: tuple indices must be integers or slices, not str

Environment

fairseq Version : main
PyTorch Version (1.10.0)
OS: Linux
How you installed fairseq (pip, source): Yes
Python version: 3.8.12
CUDA/cuDNN version: cuda_11.1
GPU models and configuration: A100

Issue Analytics

State:
Created a year ago
Comments:8 (3 by maintainers)

Top GitHub Comments

2reactions

yuntangcommented, Jun 12, 2022

{'load_pretrained_speech_text_decoder':'','load_pretrained_speech_text_encoder':''} is input for --model-overrides so you need to use quotation mark to wrap them.

0reactions

sathyagorlacommented, Jun 28, 2022

Hi @czy97 @yuntang can you provide me inference script or command I want to test my trained model (checkpoint_best.pt) with some audio files can you help me to do testing

Top Results From Across the Web

MVP: Multi-task Supervised Pre-training for Natural ...

Motivated by the success of supervised pre-training, we propose Multi-task superVised Pre-training (MVP) for natural language generation.

There was an error in evaluating the Pre-request Script

Pre-request scripts are ran before the request is sent. You do not have a response yet. Try putting your script under the Tests...

Model Training and Evaluation - Stanza - Stanford NLP Group

To train your own models, you will need to clone the source code from the stanza git repository and follow the procedures below....

Evaluation of Pretraining Methods for Deep Reinforcement ...

Reinforcement Learning models using both novel and state-of-the-art pretraining methods. Evaluation is done in Atari games, using recorded data from a.

Action error evaluation

There is no specific version for this documentation. ... Review claimed or skipped global files ... Run background scripts using VS Code.