How can we get features from wav2vec 2.0

❓ Questions and Help

Before asking:

1. search the issues.
2. search the docs.

What is your question?

Hi, for wav2vec 2.0, how can I get pre-trained features like wav2vec? https://github.com/pytorch/fairseq/tree/master/examples/wav2vec#pre-trained-models-1

My error ⬇️ /content/fairseq/fairseq/models/wav2vec/wav2vec2.py in init(self, cfg) 244 self.mask_prob = cfg.mask_prob 245 self.mask_selection = cfg.mask_selection –> 246 self.mask_other = cfg.mask_other 247 self.mask_length = cfg.mask_length 248 self.no_mask_overlap = cfg.no_mask_overlap

AttributeError: ‘Namespace’ object has no attribute ‘mask_other’

Code

import torch
from fairseq.models.wav2vec import Wav2Vec2Model

cp = torch.load('/content/pretrain_model/wav2vec_vox_new.pt')
model = Wav2Vec2Model.build_model(cp['args'], task=None)
model.load_state_dict(cp['model'])
model.eval()

wav_input_16khz = torch.randn(1,10000)
z = model.feature_extractor(wav_input_16khz)
c = model.feature_aggregator(z)

What’s your environment?

• fairseq Version (e.g., 1.0 or master):
• PyTorch Version (e.g., 1.0) 1.7.0
• OS (e.g., Linux):
• How you installed fairseq (pip, source): cd fairseq pip install --editable ./
• Build command you used (if compiling from source):
• Python version: 3.7
• CUDA/cuDNN version: 10.1

[alexeib]

can you load it like this instead:

import fairseq

cp = '/content/pretrain_model/wav2vec_vox_new.pt'
model, cfg, task = fairseq.checkpoint_utils.load_model_ensemble_and_task([cp])
model = model[0]

after that you can extract features from feature encoder in the way you tried it, or from the transformer by just doing a model forward (with mask=False and features_only=True). you may also want to try features from different layers of the model, but you need to make some changes to return results from each layer, here: https://github.com/pytorch/fairseq/blob/master/fairseq/models/wav2vec/wav2vec2.py#L764 and then also in the model forward()

[stonelazy]

Have been facing a same problem. Faced many compatibility issues. Turns out the reason was that pip source was not updated. I tried to install and develop locally. Then XLS43, wav2vec_large,wav2vec_small everything was working properly.