whisper

Whisper is OpenAIs robust multilingual speech recognition model known for accuracy across diverse conditions. **Architecture**: Encoder-decoder transformer trained on 680,000 hours of multilingual audio. Predicts text tokens from audio mel spectrograms. **Capabilities**: Transcription (speech to text in same language), translation (speech to English), language detection, timestamp generation, VAD (voice activity detection). **Multilingual**: 99 languages supported, varying quality. Strong on high-resource languages (English, Spanish, Mandarin). **Robustness**: Trained on diverse data including noisy conditions, accents, technical audio. Handles real-world audio well. **Model sizes**: Tiny (39M) to Large-v3 (1.5B). Larger models more accurate, slower. **Open source**: Weights publicly available, extensive community ecosystem. **Integrations**: Faster-whisper (4x faster), WhisperX (word-level timestamps), whisper.cpp (C++ port). **Use cases**: Transcription services, subtitle generation, voice assistants, meeting notes, accessibility. **Limitations**: Hallucination in silence, struggles with some heavy accents. **Impact**: Raised quality bar for open speech recognition, widely adopted baseline.