stopping criteria

**Stopping criteria** is the **formal set of rules used by the decoder to decide when generation should end for a request** - they define termination behavior across all decoding modes. **What Is Stopping criteria?** - **Definition**: Configured conditions that signal completion of generation. - **Criterion Types**: Length limits, EOS detection, stop strings, grammar completion, and timeout guards. - **Scope**: Applied consistently across greedy, beam, and sampling decoders. - **Implementation Point**: Evaluated each decode step before requesting next token. **Why Stopping criteria Matters** - **Consistency**: Standard criteria ensure predictable response boundaries. - **Resource Control**: Prevent runaway generation that exhausts token budgets. - **Format Reliability**: Support strict outputs like JSON and template-constrained text. - **Latency Governance**: Termination rules contribute directly to SLA compliance. - **Safety Assurance**: Stops generation when policy or execution limits are reached. **How It Is Used in Practice** - **Policy Specification**: Document stop priorities and precedence across multiple criteria. - **Edge-Case Validation**: Test nested stops, partial matches, and multilingual tokenization effects. - **Runtime Audits**: Log triggered criterion type for each completion to detect anomalies. Stopping criteria is **the termination contract of any production decoding pipeline** - robust criteria prevent truncation bugs and uncontrolled output growth.