max length

**Max length** is the **hard upper bound on generated token count for a response or completion request** - it is the primary guardrail against unbounded decoding. **What Is Max length?** - **Definition**: Configured maximum number of new tokens allowed per generation call. - **Boundary Role**: Acts as final safety cap even when other stop conditions fail. - **Interaction**: Works alongside EOS detection, stop sequences, and timeout policies. - **Deployment Context**: Commonly set per endpoint, model tier, or customer plan. **Why Max length Matters** - **Cost Control**: Caps token usage for predictable billing and infrastructure load. - **Latency Limits**: Prevents excessively long responses that violate user expectations. - **Abuse Resistance**: Reduces impact of prompts designed to force runaway generation. - **Capacity Planning**: Simplifies throughput forecasting and queue management. - **UX Consistency**: Keeps responses within expected length ranges per product surface. **How It Is Used in Practice** - **Tiered Limits**: Set different max lengths for chat, analysis, and background jobs. - **Prompt Alignment**: Pair limits with instructions to produce concise or detailed outputs. - **Monitoring**: Track truncation rates to detect limits that are too restrictive. Max length is **a mandatory control for safe and economical inference** - proper max-length policy balances completeness, cost, and latency.