acceptance rate

**Acceptance rate** is the **fraction of draft-proposed tokens that pass target-model verification during speculative decoding** - it is the primary efficiency metric for speculative inference. **What Is Acceptance rate?** - **Definition**: Ratio of accepted tokens to total proposed tokens over a decoding interval. - **Interpretation**: Higher values indicate stronger draft-target alignment and better speed potential. - **Metric Scope**: Can be measured globally, per model pair, or per traffic segment. - **Operational Link**: Directly influences effective tokens generated per expensive target-model pass. **Why Acceptance rate Matters** - **Performance Forecast**: Acceptance rate predicts practical speculative speedup more accurately than raw draft speed alone. - **Model Pair Evaluation**: Helps compare draft model candidates under real workload conditions. - **Tuning Feedback**: Reveals whether proposal length or routing policies need adjustment. - **Cost Sensitivity**: Low acceptance can increase overhead and negate expected savings. - **Stability Monitoring**: Sudden drops can indicate distribution shift or prompt drift. **How It Is Used in Practice** - **Segmented Dashboards**: Track acceptance by endpoint, prompt type, and context length bands. - **Policy Adaptation**: Dynamically shorten proposal windows when acceptance falls. - **Root-Cause Analysis**: Inspect rejection hotspots to improve draft model or prompt normalization. Acceptance rate is **the key operational KPI for speculative decoding health** - sustained high acceptance is required to realize stable inference acceleration benefits.