wav2vec

Wav2vec learns powerful speech representations through self-supervised pre-training on unlabeled audio. **Core idea**: Like BERT for audio - learn general representations from massive unlabeled audio, fine-tune for downstream tasks with small labeled data. **Wav2vec 2.0 architecture**: CNN feature encoder leads to transformer context network leads to contrastive loss. Learns to identify correct latent for masked positions from distractors. **Training**: Mask portions of audio, predict masked latent representations from negatives via contrastive learning. **Pre-training data**: Thousands of hours of unlabeled speech. **Downstream tasks**: ASR (speech recognition), speaker ID, emotion recognition, language identification. **Results**: Approaches supervised performance with 1% of labeled data. Enables ASR for low-resource languages. **XLS-R / XLSR**: Multilingual wav2vec trained on 128 languages. **HuBERT**: Alternative self-supervised approach using clustering-based targets. **Fine-tuning**: Add linear layer or CTC head, fine-tune on labeled task. **Impact**: Democratized speech AI - high-quality ASR possible without massive labeled corpora. Foundation for many speech models and APIs.