draft model selection

**Draft model selection** is the **process of choosing the proposer model used in speculative decoding to maximize acceptance rate and net speedup under quality constraints** - selection quality determines whether speculative decoding delivers real benefit. **What Is Draft model selection?** - **Definition**: Model-pairing decision that balances draft speed against proposal accuracy. - **Selection Criteria**: Includes draft latency, token agreement with target model, and serving cost. - **Compatibility Need**: Tokenizer and vocabulary alignment are required for stable verification. - **Operational Role**: Affects acceptance distribution, rejection overhead, and final throughput. **Why Draft model selection Matters** - **Speedup Realization**: Poor draft choices can erase expected speculative decoding gains. - **Cost Tradeoff**: Draft inference must be cheap enough relative to target-model savings. - **Quality Stability**: Misaligned draft behavior increases rejection churn and latency variance. - **Workload Fit**: Different domains and output styles may favor different draft models. - **Platform Efficiency**: Optimal pairing improves end-to-end tokens-per-second and SLA outcomes. **How It Is Used in Practice** - **Candidate Benchmarking**: Evaluate multiple draft models on acceptance and speed across traffic samples. - **Adaptive Routing**: Use different draft models for distinct task classes when beneficial. - **Continuous Reassessment**: Revalidate pairings after target-model or prompt-distribution changes. Draft model selection is **a key optimization step in speculative serving design** - careful proposer selection is required to convert theory-level speedups into production gains.