Beam Search and Nucleus Sampling Decoding

Beam Search and Nucleus Sampling Decoding are complementary strategies for generating high-quality text from language models by balancing diversity and quality — beam search explores most-likely paths while nucleus sampling maintains coherence through probabilistic token selection from adaptive vocabulary.

Beam Search Algorithm:
- Multiple Hypotheses: maintaining B best partial sequences (beams) sorted by cumulative log probability — B=3-5 typical with diminishing returns beyond 10
- Expansion Step: extending each beam by one token, computing softmax over 50K vocabulary — O(B*V) complexity per step where V is vocabulary size
- Pruning: keeping only top B hypotheses from B×V candidates using priority queue — reduces memory from exponential to linear in B
- Length Normalization: dividing scores by sequence length^α (α=0.6-0.7) to prevent bias toward short sentences — prevents algorithm favoring 1-2 word outputs
- Coverage Penalty: penalizing repeated coverage of same input tokens (for encoder-decoder models like T5) — improves summary diversity

Beam Search Characteristics:
- Quality Improvement: 5-10 BLEU point improvement on machine translation vs greedy (e.g., 28.0→33.5 BLEU) — noticeable in benchmarks but marginal in human evaluation
- Computational Cost: B=5 increases latency 5x due to batch processing larger number of sequences — trading generation speed for slightly better quality
- Determinism: identical outputs given same seed, reproducible across runs — useful for testing but unsuitable for creative tasks
- Hallucination Rate: 40-60% reduction in factual errors compared to greedy on QA tasks — especially beneficial for knowledge-critical applications

Nucleus (Top-P) Sampling:
- Cumulative Probability: selecting smallest vocabulary subset with cumulative probability >P (P=0.9 typical) — dynamically sized vocabulary per token
- Sorted Selection: ranking tokens by probability, accumulating until threshold P crossed — adaptive vocabulary 20-200 tokens depending on distribution
- Sampling: uniformly sampling from nucleus subset then applying temperature scaling — introduces beneficial stochasticity
- Temperature Interaction: combining nucleus (P) with temperature T for fine-grained control — P=0.9, T=0.8 balances quality and diversity

Top-K Sampling Approach:
- Fixed Vocabulary: sampling only from top K highest probability tokens (K=40-50 typical) — prevents sampling from extremely low probability tokens
- Hyperparameter Sensitivity: K=10 produces very focused outputs, K=100 allows more diversity — requires manual tuning per application
- Computational Simplicity: partial sort identifying top K requires O(Klog(V)) vs full sort O(Vlog(V)) — marginal speedup compared to nucleus
- Comparison: nucleus sampling outperforms fixed top-K on diversity while maintaining quality (human preference 65-75% in studies)

Temperature Scaling Impact:
- T=0: greedy decoding selecting arg-max token — deterministic, prone to repetition
- T=0.7: sharp distribution sharpening rare tokens, reducing diversity — recommended for factual tasks (QA, summarization)
- T=1.0: no scaling, using model calibrated probabilities — baseline setting
- T=1.5: softened distribution emphasizing diversity — recommended for creative tasks (story generation, dialogue)

Practical Decoding Strategies:
- Repetition Penalty: dividing logit of previously generated tokens by penalty parameter (1.0-2.0) — prevents repetitive sequences common in nucleus sampling
- Length Penalty: decreasing future token logits as sequence grows — encourages longer generations (useful for minimum length requirements)
- Bad Words Filter: zeroing logits of inappropriate tokens before sampling — prevents toxic or off-topic outputs
- Constraint Satisfaction: modifying probabilities to steer toward particular semantic constraints (CommonSense reasoning, QA answer format)

Beam Search and Nucleus Sampling Decoding are complementary techniques — beam search providing quality improvements for deterministic tasks while nucleus sampling enables creative, diverse text generation for conversational and creative applications.