type-constrained decoding

**Type-constrained decoding** is a structured generation technique that ensures LLM outputs conform to specified **data types and type structures** — such as integers, floats, booleans, enums, lists of specific types, or complex nested objects. It provides type safety for LLM outputs, similar to type checking in programming languages. **How It Works** - **Type Specification**: The developer defines the expected output type using a **type system** — this could be Python type hints, TypeScript types, JSON Schema, or Pydantic models. - **Grammar Generation**: The type specification is automatically converted into a **formal grammar** or set of token constraints. - **Constrained Sampling**: During generation, only tokens valid for the current type context are permitted. **Type Constraint Examples** - **Primitive Types**: `int` → only digits (and optional sign); `bool` → only "true" or "false"; `float` → digits with decimal point. - **Enum Types**: `Literal["small", "medium", "large"]` → only these exact strings. - **Composite Types**: `List[int]` → a JSON array containing only integers; `Dict[str, float]` → a JSON object with string keys and float values. - **Complex Objects**: Pydantic models or dataclasses with nested typed fields. **Frameworks and Tools** - **Outlines**: Supports Pydantic models and JSON Schema for type-constrained generation. - **Instructor**: Library by Jason Liu that adds type-constrained outputs to OpenAI and other LLM APIs using Pydantic models. - **Marvin**: Type-safe AI function calls with Python type hints. - **LangChain Structured Output**: Provides type-constrained output parsing with retry logic. **Benefits** - **Eliminates Parsing Errors**: Output is guaranteed to be parseable into the target type. - **Developer Experience**: Define expected types once using familiar type systems, and the framework handles constraint enforcement. - **Composability**: Complex types are built from simpler ones, matching natural programming patterns. Type-constrained decoding represents the maturation of LLM integration — treating model outputs as **typed data** rather than unpredictable strings.