Speech-to-Text with Whisper

What is Whisper? OpenAI Whisper is an automatic speech recognition (ASR) model trained on 680,000 hours of multilingual audio.

Model Sizes

Basic Usage

import whisper

model = whisper.load_model("base")
result = model.transcribe("audio.mp3")

print(result["text"])

Advanced Options

result = model.transcribe(
    "audio.mp3",
    language="en",           # Specify language
    task="transcribe",       # or "translate"
    word_timestamps=True,    # Per-word timing
    fp16=True               # Faster on GPU
)

# Word-level timestamps
for segment in result["segments"]:
    for word in segment["words"]:
        print(f"{word['start']:.2f}s: {word['word']}")

Faster Whisper Optimized implementation using CTranslate2:

from faster_whisper import WhisperModel

model = WhisperModel("large-v3", compute_type="float16")
segments, info = model.transcribe("audio.mp3")

for segment in segments:
    print(f"[{segment.start:.2f}s] {segment.text}")

Real-Time Transcription

import sounddevice as sd
import queue

audio_queue = queue.Queue()

def callback(indata, frames, time, status):
    audio_queue.put(indata.copy())

# Stream and transcribe chunks
with sd.InputStream(callback=callback):
    while True:
        audio_chunk = audio_queue.get()
        text = model.transcribe(audio_chunk)["text"]
        print(text, end=" ", flush=True)

Use Cases

Speaker Diarization Identify who is speaking:

from pyannote.audio import Pipeline

diarization = Pipeline.from_pretrained("pyannote/speaker-diarization")
result = diarization("audio.wav")

for turn, _, speaker in result.itertracks(yield_label=True):
    print(f"{turn.start:.1f}s - {turn.end:.1f}s: {speaker}")

Best Practices

• Use larger models for noisy audio
• Specify language when known
• Combine with diarization for meetings
• Consider Faster Whisper for speed
• Chunk long audio for memory efficiency