Guidance is a Microsoft-developed programming language for constraining and controlling LLM outputs with guaranteed structure — replacing probabilistic prompt engineering with deterministic template execution that interleaves generation and computation, ensuring the model produces exactly the format (JSON, XML, code, structured dialogue) your application needs without relying on post-hoc parsing or retry loops.

What Is Guidance?

• Definition: An open-source Python library from Microsoft that uses a Handlebars-inspired template syntax to precisely control LLM generation — mixing static text, conditional logic, loops, and constrained generation directives in a single coherent template.
• The Core Problem: Standard prompt engineering asks the LLM nicely to output a specific format ("Please respond in JSON"). The model often refuses, adds extra text, or subtly breaks the schema. Guidance enforces the format at the token level.
• Constrained Generation: Using {{gen}}, {{select}}, and {{regex}} directives, Guidance modifies the logits during sampling — making it physically impossible for the model to deviate from the specified structure.
• Interleaved Execution: Templates mix pre-written text, Python computation, and LLM generation — a template can call Python functions mid-generation, use their results to condition subsequent generation, and produce complex structured outputs in a single pass.
• Efficiency: By constraining generation and reusing prompt prefixes (via KV-cache), Guidance reduces token waste and latency compared to generate-parse-retry loops.

Why Guidance Matters

• Reliability: Applications that need structured output (JSON APIs, form extraction, classification) gain 100% format compliance without retry logic — the model cannot produce malformed output.
• Reduced Latency: A single guided generation pass replaces the generate→parse→retry cycle that can require 3-5 LLM calls for complex structured outputs.
• Complex Logic: Conditional generation ({{#if condition}}...{{/if}}), loops ({{#each items}}), and branching enable structured dialogues and decision trees that would be impossible with standard prompting.
• Local Model Optimization: Guidance is particularly powerful with local models (Llama, Mistral) where you control the inference stack — enabling grammar-constrained generation at the token level.
• Microsoft Production Use: Used internally at Microsoft for structured data extraction from documents, multi-turn dialogue systems, and code generation pipelines.

Guidance Template Syntax

Basic Constrained Generation:

import guidance
lm = guidance.models.OpenAI("gpt-4")

with guidance.system():
    lm += "You extract information from text."
with guidance.user():
    lm += "Extract the city from: I live in Paris, France."
with guidance.assistant():
    lm += "City: " + guidance.gen("city", stop=".")

Select Directive — forces the model to choose from a fixed list:

lm += "Sentiment: " + guidance.select(["positive", "negative", "neutral"], name="sent")

Regex Constraint — ensures output matches a pattern:

lm += "Date: " + guidance.gen("date", regex=r"d{4}-d{2}-d{2}")

Key Guidance Directives

• {{gen name}}: Generate text and capture it as a named variable for downstream use.
• {{select name options=[...]}}: Force selection from a discrete set — zero probability for non-listed tokens.
• {{regex pattern}}: Constrain generation to match a regular expression exactly.
• {{#if variable}}: Conditional template blocks based on previously generated or Python-computed values.
• {{#each items}}: Loop over a list, generating structured output for each item.

Guidance vs Alternatives

Use Cases

• Structured Data Extraction: Extract named entities, dates, and relationships from documents into guaranteed-valid JSON.
• Classification Pipelines: Multi-label classification with forced selection from taxonomy — no hallucinated categories.
• Dialogue Systems: Multi-turn conversations where each turn follows a specific schema — useful for intake forms, troubleshooting trees, and customer service bots.
• Code Generation: Generate code blocks within a larger structured response that includes documentation, type signatures, and test cases.

Guidance is the deterministic alternative to probabilistic prompt engineering — for applications where structured output is non-negotiable, Guidance replaces fragile "please format as JSON" instructions with guaranteed, token-level constrained generation that eliminates the entire class of output parsing failures.