speech recognition

**Automatic Speech Recognition (ASR)** is the **task of converting spoken audio to text** — transforming acoustic waveforms into transcribed words, enabling voice assistants, transcription services, real-time captioning, and spoken language interfaces. **ASR Pipeline** **Traditional**: 1. Feature extraction: MFCC (Mel Frequency Cepstral Coefficients) or log-Mel spectrogram. 2. Acoustic model: GMM-HMM → predicts phonemes. 3. Language model: N-gram → refines word sequence. 4. Decoder: Beam search combining acoustic + language scores. **End-to-End Deep Learning**: - Entire pipeline replaced by single neural network. - Input: Raw audio or log-Mel spectrogram. - Output: Character/subword sequence directly. **CTC (Connectionist Temporal Classification)** - Enables end-to-end training without alignment between audio and text. - CTC loss: Marginalizes over all valid alignments of output to target. - Key innovation: Blank token handles repeated characters and silence. - Used in: Deep Speech, QuartzNet, Citrinet. **Attention-Based Encoder-Decoder** - Encoder: Processes audio features (CNN + LSTM or Transformer). - Decoder: Attends to encoder output to generate transcript token by token. - RNN-T (Recurrent Neural Transducer): CTC + attention — better for streaming. **Whisper (OpenAI, 2022)** - 680K hours of multilingual training data — largest public ASR dataset. - Architecture: Transformer encoder-decoder with 80-dim log-Mel spectrogram. - Capabilities: Transcription, translation (non-English → English), language detection, timestamps. - Sizes: Tiny (39M) to Large-v3 (1.55B). - Whisper Large: 2.7% WER on LibriSpeech clean — near human level. **wav2vec 2.0 / HuBERT** - Self-supervised pretraining on unlabeled audio. - Contrastive learning over quantized speech representations. - Fine-tuned on small labeled datasets → strong low-resource ASR. ASR is **a solved problem for high-resource languages under clean conditions** — the remaining frontier is low-resource languages, domain-specific vocabulary, noisy environments, and real-time on-device inference where Whisper distillation and streaming models continue to advance rapidly.