length penalty

Length penalty adjusts sequence scores in beam search to control output length preferences. **Problem**: Log probabilities accumulate negatively - longer sequences have lower scores, biasing toward short outputs. **Solution**: Normalize by length: score = log_prob / length^α. **Alpha values**: α = 0 (no normalization, favor short), α = 1 (linear normalization), α > 1 (favor longer sequences), α < 1 (mild length compensation). **Google's formula**: lp(Y) = ((5 + |Y|)/(5 + 1))^α - smoothed length penalty avoiding division by zero for short sequences. **Implementation**: Apply during beam selection and final ranking. **Use cases**: Translation (α ≈ 0.6-0.8 for balanced length), summarization (adjust based on desired length), structured outputs. **Related controls**: Max/min length constraints, length-conditional training and sampling. **Alternatives**: Direct length tokens in prompt, length-conditioned decoding, explicit length prediction. **Best practices**: Tune α empirically on validation set, different tasks need different settings, combine with other quality metrics for final selection.