length normalization

**Length normalization** is the **score-adjustment technique that compensates for sequence-length bias when ranking generated hypotheses in search-based decoding** - it prevents unfair preference for overly short outputs. **What Is Length normalization?** - **Definition**: Normalization of cumulative log-probability scores by sequence length or related scaling formulas. - **Bias Correction**: Raw likelihood sums naturally penalize longer sequences, requiring correction for fair comparison. - **Decoding Context**: Commonly applied in beam search and other hypothesis-ranking methods. - **Parameter Role**: Normalization strength controls balance between brevity and completeness. **Why Length normalization Matters** - **Answer Completeness**: Without normalization, decoders can truncate before fully answering queries. - **Quality Ranking**: Improves selection fairness across hypotheses of different lengths. - **Task Fit**: Critical for translation, summarization, and QA where output length varies naturally. - **User Satisfaction**: Reduces clipped or underspecified responses in production assistants. - **Evaluation Alignment**: Better hypothesis ranking improves downstream quality metrics. **How It Is Used in Practice** - **Formula Selection**: Choose normalization function suited to task and model behavior. - **Hyperparameter Tuning**: Sweep normalization strength on held-out datasets. - **Failure Analysis**: Inspect too-short and too-long outputs to recalibrate scoring balance. Length normalization is **a necessary correction for length-biased search scoring** - proper normalization improves completeness without sacrificing ranking quality.