speculative decoding

**Speculative Decoding** is the **inference acceleration technique that uses a smaller, faster draft model to propose multiple tokens in parallel, which the larger target model then verifies in a single forward pass** — exploiting the fact that verification of N tokens (one forward pass through the target) is much cheaper than generating N tokens autoregressively (N forward passes), achieving 2-3× speedup with mathematically guaranteed identical output distribution to the original model, making it one of the few "free lunch" optimizations for LLM inference. **The Autoregressive Bottleneck** ``` Standard autoregression (100 tokens): Token 1 → [Full model forward pass] → Token 2 → [Full model forward pass] → ... 100 sequential forward passes, each memory-bandwidth-bound Time: 100 × latency_per_token Speculative decoding (100 tokens): Draft model proposes K tokens in parallel Target model verifies K tokens in one forward pass Accept all correct tokens, regenerate from first wrong one Time: ~(100/K) × latency_per_token (if acceptance rate is high) ``` **How It Works** ``` 1. Draft model generates K candidate tokens: [The] → draft → [quick] [brown] [fox] [jumped] [over] 2. Target model scores ALL candidates in one forward pass: P_target(quick|The) = 0.85 (draft said 0.80) → Accept P_target(brown|The quick) = 0.90 (draft said 0.88) → Accept P_target(fox|...brown) = 0.75 (draft said 0.70) → Accept P_target(jumped|...fox) = 0.30 (draft said 0.60) → Reject! 3. Accept first 3 tokens, resample token 4 from adjusted distribution Output: [The] [quick] [brown] [fox] [leaped] Net gain: 3 tokens verified in 1 target pass instead of 3 passes ``` **Mathematical Guarantee** - Acceptance criterion uses modified rejection sampling. - If P_draft(x) ≤ P_target(x): Always accept. - If P_draft(x) > P_target(x): Accept with probability P_target(x)/P_draft(x). - On rejection: Sample from residual distribution (P_target - P_draft). - Theorem: Output distribution is exactly P_target regardless of draft model quality. **Draft Model Strategies** | Strategy | Draft Model | Overhead | Acceptance Rate | |----------|------------|---------|----------------| | Smaller same-family | Llama-3-8B drafts for Llama-3-70B | Low | 70-85% | | Quantized self | INT4 version of target | Minimal | 75-90% | | Early exit | First N layers of target | Minimal | 60-80% | | Medusa heads | MLP heads on target model | Very low | 60-75% | | Eagle | Feature-level autoregressive draft | Low | 75-85% | | N-gram / retrieval | Statistical lookup | Near zero | 40-60% | **Performance Results** | Setup | Speedup | Use Case | |-------|---------|----------| | 7B drafts for 70B | 2.0-2.5× | General text generation | | Medusa heads | 2.0-2.8× | No separate draft model needed | | Eagle-2 | 2.5-3.5× | Best draft architecture | | Self-speculative (early exit) | 1.5-2.0× | Simplest to deploy | **When Speculative Decoding Helps Most** - Batch size 1 (interactive): Maximum benefit (memory-bandwidth bound). - Code generation: High acceptance rate (code is predictable). - Translation: Draft model easily approximates structure. - Large batch: Less benefit (compute-bound, not bandwidth-bound). Speculative decoding is **the most important inference optimization for interactive LLM serving** — by turning the sequential token-generation bottleneck into a parallel verify-and-accept loop, speculative decoding delivers 2-3× latency reduction with zero quality degradation, making it essential infrastructure for real-time AI applications from chatbots to code assistants, where every millisecond of response time directly impacts user experience.

Go deeper with CFSGPT

Get AI-powered deep-dives, save terms, and run advanced simulations — free account.

Create Free Account