constrained beam search

**Constrained beam search** is a decoding algorithm that extends standard **beam search** with additional constraints that the generated output must satisfy. It explores multiple candidate sequences simultaneously while enforcing structural, formatting, or content requirements on the final output. **How Standard Beam Search Works** - Maintains **k candidate sequences** (beams) at each generation step. - At each step, expands each beam with all possible next tokens, scores them, and keeps the top **k** overall candidates. - Returns the highest-scoring complete sequence. **Adding Constraints** - **Format Constraints**: Force output to follow specific patterns — valid JSON, XML, or structured data formats. - **Lexical Constraints**: Require certain words or phrases to appear in the output (e.g., "the answer must contain 'TSMC'"). - **Length Constraints**: Enforce minimum or maximum output length. - **Vocabulary Constraints**: Restrict generation to a subset of the vocabulary at each step. **Implementation Approaches** - **Token Masking**: At each step, compute which tokens violate constraints and set their probabilities to zero (or negative infinity in log space) before beam selection. - **Grid Beam Search**: Tracks constraint satisfaction state alongside sequence state, using a **multi-dimensional beam** that progresses through both sequence position and constraint fulfillment. - **Bank-Based Methods**: Organize beams into "banks" based on how many constraints have been satisfied, ensuring diverse constraint coverage. **Trade-Offs** - **Quality vs. Control**: More constraints reduce the search space, potentially forcing lower-quality text to satisfy requirements. - **Computational Cost**: Constraint checking at each step adds overhead, and complex constraints may require significantly more beams. - **Guarantee Level**: Depending on implementation, constraints can be **hard** (always satisfied) or **soft** (preferred but not guaranteed). **Applications** Constrained beam search is used in **machine translation** (terminology enforcement), **data-to-text generation** (ensure all facts are mentioned), **structured output generation**, and any scenario where outputs must comply with predefined rules.