whisper
**Speech-to-Text with Whisper**
**What is Whisper?**
OpenAI Whisper is an automatic speech recognition (ASR) model trained on 680,000 hours of multilingual audio.
**Model Sizes**
| Model | Parameters | Speed | Quality |
|-------|------------|-------|---------|
| tiny | 39M | Fastest | Basic |
| base | 74M | Fast | Good |
| small | 244M | Medium | Better |
| medium | 769M | Slow | Great |
| large-v3 | 1.5B | Slowest | Best |
**Basic Usage**
```python
import whisper
model = whisper.load_model("base")
result = model.transcribe("audio.mp3")
print(result["text"])
```
**Advanced Options**
```python
result = model.transcribe(
"audio.mp3",
language="en", # Specify language
task="transcribe", # or "translate"
word_timestamps=True, # Per-word timing
fp16=True # Faster on GPU
)
# Word-level timestamps
for segment in result["segments"]:
for word in segment["words"]:
print(f"{word['start']:.2f}s: {word['word']}")
```
**Faster Whisper**
Optimized implementation using CTranslate2:
```python
from faster_whisper import WhisperModel
model = WhisperModel("large-v3", compute_type="float16")
segments, info = model.transcribe("audio.mp3")
for segment in segments:
print(f"[{segment.start:.2f}s] {segment.text}")
```
**Real-Time Transcription**
```python
import sounddevice as sd
import queue
audio_queue = queue.Queue()
def callback(indata, frames, time, status):
audio_queue.put(indata.copy())
# Stream and transcribe chunks
with sd.InputStream(callback=callback):
while True:
audio_chunk = audio_queue.get()
text = model.transcribe(audio_chunk)["text"]
print(text, end=" ", flush=True)
```
**Use Cases**
| Use Case | Features Needed |
|----------|-----------------|
| Meeting transcription | Timestamps, speaker diarization |
| Podcast processing | Chapter markers |
| Subtitles | Word timestamps, formatting |
| Voice search | Fast, streaming |
| Accessibility | High accuracy |
**Speaker Diarization**
Identify who is speaking:
```python
from pyannote.audio import Pipeline
diarization = Pipeline.from_pretrained("pyannote/speaker-diarization")
result = diarization("audio.wav")
for turn, _, speaker in result.itertracks(yield_label=True):
print(f"{turn.start:.1f}s - {turn.end:.1f}s: {speaker}")
```
**Best Practices**
- Use larger models for noisy audio
- Specify language when known
- Combine with diarization for meetings
- Consider Faster Whisper for speed
- Chunk long audio for memory efficiency