outlines
**Outlines** is the **open-source structured generation library that uses finite state machines and grammar-based constraints to guarantee LLM outputs conform to specified schemas** — enabling reliable JSON generation, regex-constrained text, and type-safe outputs by restricting the model's token sampling to only valid continuations at each generation step.
**What Is Outlines?**
- **Definition**: A Python library for structured text generation that compiles output specifications (JSON schemas, regex patterns, grammars) into token-level constraints applied during LLM decoding.
- **Core Innovation**: Uses finite state machines (FSMs) and context-free grammars to compute valid next tokens at each step, guaranteeing structural correctness.
- **Key Difference**: Operates at the token sampling level — invalid tokens are masked before sampling, making malformed output impossible.
- **Creator**: dottxt (formerly .txt), open-source community.
**Why Outlines Matters**
- **100% Structure Compliance**: Every generated output is guaranteed valid — no parsing errors, no retries needed.
- **Efficient**: Constraint compilation happens once; per-token masking adds minimal overhead during generation.
- **Flexible Constraints**: JSON Schema, regex, context-free grammars, Python type hints, and Pydantic models.
- **Model Agnostic**: Works with any model supporting logit manipulation (Hugging Face, vLLM, llama.cpp).
- **Open Source**: Fully open with active community development and integration ecosystem.
**Core Constraint Types**
| Constraint | Input | Guarantee |
|------------|-------|-----------|
| **JSON Schema** | Pydantic model or JSON Schema | Valid JSON matching schema |
| **Regex** | Regular expression pattern | Output matches pattern exactly |
| **Grammar** | Context-free grammar (BNF/EBNF) | Syntactically valid output |
| **Choice** | List of valid options | Output is one of the specified choices |
| **Type** | Python type (int, float, bool) | Correctly typed output |
**How Outlines Works**
1. **Compile**: Convert the output specification (JSON Schema, regex) into a finite state machine.
2. **Index**: Pre-compute which vocabulary tokens are valid transitions from each FSM state.
3. **Generate**: At each generation step, mask invalid tokens before sampling the next token.
4. **Guarantee**: The FSM ensures the complete output satisfies the specification.
**Integration Ecosystem**
- **vLLM**: High-throughput structured generation for production serving.
- **Hugging Face**: Direct integration with Transformers models.
- **llama.cpp**: Local inference with structured output.
- **LangChain/LlamaIndex**: Use as output parser in RAG pipelines.
Outlines is **the gold standard for guaranteed structured LLM output** — solving the fundamental reliability problem of language model generation through mathematical guarantees rather than probabilistic hoping, making it essential for production systems requiring strict output compliance.