Logit bias manually adjusts token probabilities before sampling to encourage or suppress specific outputs. Mechanism: Add (or subtract) fixed values to logits of specified tokens before softmax. Positive bias → more likely, negative bias → less likely, -100 effectively bans token. Use cases: Ensure specific format tokens appear, prevent problematic terms, guide structured generation, enforce vocabulary constraints. API support: OpenAI API accepts token ID → bias value dictionary, other providers have similar features. Examples: Ban curse words (negative bias), encourage JSON formatting tokens, suppress competitor names, ensure answer ends with period. Relationship to prompting: Complements instructions - bias provides hard constraints, prompts give soft guidance. Tokens to bias: Use tokenizer to find exact token IDs - be aware of multi-token words. Trade-offs: Can create awkward outputs if overused, may interfere with natural generation, requires knowing exact token IDs. Best practices: Use sparingly for critical constraints, test thoroughly, prefer prompting for soft preferences, save hard constraints for format-critical applications.
Explore 500+ Semiconductor & AI Topics
From EUV lithography to CUDA optimization — search the full knowledge base or chat with our AI assistant.