contrastive decoding
**LLM Decoding Strategies** are the **algorithms that determine how tokens are selected from a language model's probability distribution during text generation** — ranging from deterministic methods like greedy and beam search to stochastic approaches like nucleus (top-p) sampling and temperature scaling, and advanced methods like contrastive decoding that exploit differences between strong and weak models, where the choice of decoding strategy profoundly affects output quality, diversity, coherence, and factuality.
**Decoding Methods Overview**
| Method | Type | Diversity | Quality | Speed |
|--------|------|----------|---------|-------|
| Greedy | Deterministic | None | Repetitive | Fastest |
| Beam search | Deterministic | Low | High for short | Slow |
| Top-k sampling | Stochastic | Medium | Good | Fast |
| Top-p (nucleus) | Stochastic | Medium-high | Good | Fast |
| Temperature sampling | Stochastic | Adjustable | Varies | Fast |
| Contrastive decoding | Hybrid | Medium | Very high | 2× cost |
| Min-p sampling | Stochastic | Adaptive | Good | Fast |
| Typical sampling | Stochastic | Medium | Good | Fast |
**Temperature Scaling**
```python
def temperature_sample(logits, temperature=1.0):
"""Lower temp = more confident/deterministic
Higher temp = more random/creative"""
scaled = logits / temperature
probs = softmax(scaled)
return sample(probs)
# temperature=0.0: Greedy (argmax)
# temperature=0.3: Focused, factual responses
# temperature=0.7: Balanced (common default)
# temperature=1.0: Original distribution
# temperature=1.5: Very creative, sometimes incoherent
```
**Top-p (Nucleus) Sampling**
```python
def top_p_sample(logits, p=0.9):
"""Sample from smallest set of tokens with cumulative prob >= p"""
sorted_probs, sorted_indices = torch.sort(softmax(logits), descending=True)
cumulative_probs = torch.cumsum(sorted_probs, dim=-1)
# Remove tokens with cumulative probability above threshold
sorted_probs[cumulative_probs > p] = 0
sorted_probs[0] = max(sorted_probs[0], 1e-8) # keep at least top-1
# Renormalize and sample
sorted_probs /= sorted_probs.sum()
return sample(sorted_probs)
# p=0.1: Very focused (often 1-3 tokens)
# p=0.9: Standard (typically 10-100 tokens in nucleus)
# p=1.0: Full distribution (= temperature sampling only)
```
**Contrastive Decoding**
```
Idea: Amplify what a STRONG model knows that a WEAK model doesn't
score(token) = log P_large(token) - α × log P_small(token)
Intuition:
- Both models predict common tokens similarly → low contrast
- Large model uniquely confident about factual/coherent tokens → high contrast
- Result: Suppresses generic/repetitive tokens, promotes informative ones
Effect: Significantly reduces hallucination and repetition
```
**Min-p Sampling**
```python
def min_p_sample(logits, min_p=0.05):
"""Keep tokens with probability >= min_p × max_probability"""
probs = softmax(logits)
threshold = min_p * probs.max()
probs[probs < threshold] = 0
probs /= probs.sum()
return sample(probs)
# Advantage over top-p: Adapts to distribution shape
# Confident prediction (one 90% token): min-p keeps very few tokens
# Uncertain prediction (many ~5% tokens): min-p keeps many tokens
```
**Recommended Settings by Task**
| Task | Temperature | Top-p | Strategy |
|------|-----------|-------|----------|
| Code generation | 0.0-0.2 | 0.9 | Near-greedy, correctness matters |
| Factual Q&A | 0.0-0.3 | 0.9 | Low temp for accuracy |
| Creative writing | 0.7-1.0 | 0.95 | Higher diversity |
| Chat/conversation | 0.5-0.7 | 0.9 | Balanced |
| Translation | 0.0-0.1 | — | Beam search or greedy |
| Brainstorming | 0.9-1.2 | 0.95 | Maximum diversity |
**Repetition Penalties**
- Frequency penalty: Reduce probability proportional to how often token appeared.
- Presence penalty: Fixed reduction if token appeared at all.
- Repetition penalty (multiplier): Divide logit by penalty factor for repeated tokens.
- These fix the degenerate repetition common in greedy/beam search.
LLM decoding strategies are **the often-overlooked lever that dramatically affects generation quality** — the same model can produce boring, repetitive text with greedy decoding or creative, diverse text with tuned sampling, and advanced methods like contrastive decoding can reduce hallucination by 30-50%, making decoding configuration as important as model selection for production AI systems.