Constrained beam search is a decoding algorithm that extends standard beam search with additional constraints that the generated output must satisfy. It explores multiple candidate sequences simultaneously while enforcing structural, formatting, or content requirements on the final output.

How Standard Beam Search Works

• Maintains k candidate sequences (beams) at each generation step.
• At each step, expands each beam with all possible next tokens, scores them, and keeps the top k overall candidates.
• Returns the highest-scoring complete sequence.

Adding Constraints

• Format Constraints: Force output to follow specific patterns — valid JSON, XML, or structured data formats.
• Lexical Constraints: Require certain words or phrases to appear in the output (e.g., "the answer must contain 'TSMC'").
• Length Constraints: Enforce minimum or maximum output length.
• Vocabulary Constraints: Restrict generation to a subset of the vocabulary at each step.

Implementation Approaches

• Token Masking: At each step, compute which tokens violate constraints and set their probabilities to zero (or negative infinity in log space) before beam selection.
• Grid Beam Search: Tracks constraint satisfaction state alongside sequence state, using a multi-dimensional beam that progresses through both sequence position and constraint fulfillment.
• Bank-Based Methods: Organize beams into "banks" based on how many constraints have been satisfied, ensuring diverse constraint coverage.

Trade-Offs

• Quality vs. Control: More constraints reduce the search space, potentially forcing lower-quality text to satisfy requirements.
• Computational Cost: Constraint checking at each step adds overhead, and complex constraints may require significantly more beams.
• Guarantee Level: Depending on implementation, constraints can be hard (always satisfied) or soft (preferred but not guaranteed).

Applications

Constrained beam search is used in machine translation (terminology enforcement), data-to-text generation (ensure all facts are mentioned), structured output generation, and any scenario where outputs must comply with predefined rules.