contrastive search

**Contrastive search** is the **decoding strategy that combines model confidence with degeneration penalties to select tokens that are both likely and diverse from recent context** - it is designed to reduce repetitive loops in text generation. **What Is Contrastive search?** - **Definition**: Hybrid decoding criterion balancing probability maximization and diversity-aware penalties. - **Mechanism**: Selects token candidates from top probability set, then re-ranks with similarity penalties. - **Degeneration Control**: Discourages repetitive or self-similar continuations. - **Output Style**: Typically more coherent than high-randomness sampling and less repetitive than greedy. **Why Contrastive search Matters** - **Repetition Reduction**: Penalty terms directly target common degeneration patterns. - **Quality Balance**: Maintains fluency while improving informational novelty. - **Deterministic Behavior**: Often more stable than purely stochastic sampling methods. - **Long-Form Utility**: Useful for paragraph-length outputs where repetition risk is higher. - **Operational Simplicity**: Single search routine can replace complex sampling stacks for some workloads. **How It Is Used in Practice** - **Candidate Set Size**: Tune top candidate pool for balance between quality and compute. - **Penalty Strength**: Adjust similarity penalty to avoid both repetition and incoherent jumps. - **Workload Validation**: Benchmark on long answers, summaries, and dialogue continuity tasks. Contrastive search is **a practical decoding method for fluent and less repetitive output** - contrastive search improves text quality by coupling confidence with anti-degeneration signals.