text to speech

**Neural Text-to-Speech (TTS)** is the **synthesis of natural-sounding speech from text using deep learning** — producing human-quality voice output that is indistinguishable from real speech for most applications, enabling voice assistants, audiobooks, accessibility tools, and synthetic media. **TTS Pipeline** 1. **Text Normalization**: "2.5kg" → "two point five kilograms". 2. **Text-to-Acoustic Features**: Text → mel spectrogram (acoustic model). 3. **Vocoder**: Mel spectrogram → waveform. **Acoustic Models** **Tacotron 2 (Google, 2018)**: - Seq2seq with attention: Encoder processes text characters; decoder generates mel frames. - First end-to-end TTS to achieve near-human quality. - MOS (Mean Opinion Score): 4.53/5.0 vs. 4.58 for human speech. **FastSpeech 2 (Microsoft, 2020)**: - Non-autoregressive: Parallel mel generation — 30x faster than Tacotron 2. - Duration predictor: Explicitly predicts how many mel frames per phoneme. - Variance adaptor: Controls pitch, energy, duration. **Vocoders** - **WaveNet (DeepMind, 2016)**: Dilated causal convolution, 24 kHz audio. 0.5 RTF — too slow for production. - **HiFi-GAN**: GAN-based vocoder. Real-time (RTF < 0.01), high quality. Standard in production. - **WaveGrad / DiffWave**: Diffusion-based vocoders — highest quality but slower. **End-to-End TTS** - **VITS (2021)**: Combines acoustic model + vocoder end-to-end with variational inference. - Single model: Text → waveform. No two-stage pipeline. - Naturalness competitive with two-stage at much simpler training. **Modern LLM-Based TTS** - **VoiceBox (Meta, 2023)**: Flow Matching-based, in-context voice cloning. - **Tortoise TTS**: DALL-E-like autoregressive + DDPM — ultra-high quality, slow. - **ElevenLabs, Bark**: LLM-based voice synthesis with emotion and style control. Neural TTS has **effectively solved conversational-quality voice synthesis** — the remaining challenges are real-time performance on edge devices, multilingual support without accent artifacts, and emotion expressiveness that matches the full range of human speech prosody.