lookahead decoding

**Lookahead decoding** is an **inference acceleration technique that generates multiple tokens in parallel using speculative execution** — predicting future tokens speculatively and verifying them to reduce effective latency. **What Is Lookahead Decoding?** - **Definition**: Generate and verify multiple tokens per forward pass. - **Method**: Speculate future tokens, verify in parallel. - **Speed**: 2-4× faster than standard autoregressive decoding. - **Exactness**: Produces identical output to greedy decoding. - **Requirement**: No additional models needed (unlike speculative decoding). **Why Lookahead Decoding Matters** - **Latency**: Reduces time-to-first-token and overall generation time. - **No Extra Models**: Works with single model (vs speculative decoding). - **Exact**: Guaranteed same output as standard decoding. - **LLM Inference**: Critical for production deployments. - **Cost**: More compute per step but fewer steps total. **How It Works** 1. **Speculate**: Generate n-gram candidates for future positions. 2. **Verify**: Check all candidates in single forward pass. 3. **Accept**: Keep verified tokens, discard wrong speculations. 4. **Repeat**: Continue with accepted tokens. **Comparison** - **Autoregressive**: 1 token per forward pass. - **Speculative**: Draft model + verify (needs 2 models). - **Lookahead**: Self-speculate + verify (single model). Lookahead decoding achieves **faster LLM inference without auxiliary models** — practical acceleration technique.