Stuck on an issue?
Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.
And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.
wav2vec 2.0 doesn't appear to do vector quantization
See original GitHub issueIn the paper from FAIR, they describe wav2vec 2.0 as using a vector quantization module to learn discrete vectors of speech units (section 2.) As far as I know, this should be happening between Wav2Vec2FeatureExtractor and Wav2Vec2FeatureProjection.
The HuggingFace implementation doesn’t seem to do any vector quantization. Is this a correct implementation?
Issue Analytics
- State:
- Created 2 years ago
- Comments:6 (4 by maintainers)
Top Results From Across the Web
An Illustrated Tour of Wav2vec 2.0 | Jonathan Bgn
There are four important elements in this diagram: the feature encoder, context network, quantization module, and the contrastive loss (pre- ...
Read more >Wav2Vec2 - Hugging Face
wav2vec 2.0 masks the speech input in the latent space and solves a contrastive task defined over a quantization of the latent representations...
Read more >Wav2Vec 2.0: Self-Supervised Learning for ASR
To create a quantized representation, the best word from every codebook should be selected. Then, the chosen vectors are concatenated and processed with...
Read more >arXiv:2103.08393v2 [eess.AS] 23 Jun 2021
nique combining elements from wav2vec 2.0 and VQ-VAE. Our model learns to reproduce quantized representations from par-.
Read more >wav2vec 2.0: A Framework for Self-Supervised ... - YouTube
In this tutorial i will explain the paper " wav2vec 2.0 : A Framework for Self-Supervised Learning of Speech Representations" by Alexei ...
Read more >
Top Related Medium Post
No results found
Top Related StackOverflow Question
No results found
Troubleshoot Live Code
Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free
Top Related Reddit Thread
No results found
Top Related Hackernoon Post
No results found
Top Related Tweet
No results found
Top Related Dev.to Post
No results found
Top Related Hashnode Post
No results found
The port didn’t include the quantization module - we should re-port the model 😃
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.