Structured Output and JSON Mode

Why Structured Output? LLMs naturally produce free-form text. For programmatic use, we need reliable structured output (JSON, XML, etc.).

OpenAI JSON Mode

Basic JSON Mode

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{
        "role": "user",
        "content": "Extract name and age from: John is 30 years old"
    }],
    response_format={"type": "json_object"}
)

data = json.loads(response.choices[0].message.content)
# {"name": "John", "age": 30}

Structured Outputs with Schema

from pydantic import BaseModel

class Person(BaseModel):
    name: str
    age: int
    occupation: str | None = None

response = client.beta.chat.completions.parse(
    model="gpt-4o",
    messages=[...],
    response_format=Person
)

person = response.choices[0].message.parsed
print(person.name)  # Typed access

Instructor Library Popular library for structured outputs with any LLM:

import instructor
from pydantic import BaseModel

client = instructor.from_openai(OpenAI())

class UserInfo(BaseModel):
    name: str
    age: int
    email: str

user = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Extract: John, 30, [email protected]"}],
    response_model=UserInfo
)

print(user.name)  # "John"

Outlines (for local models) Constrained generation ensuring valid JSON:

from outlines import models, generate

model = models.transformers("meta-llama/Llama-2-7b-hf")

schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "age": {"type": "integer"}
    },
    "required": ["name", "age"]
}

generator = generate.json(model, schema)
result = generator("Extract from: John is 30")

Validation Always validate LLM JSON output:

from pydantic import ValidationError

try:
    data = json.loads(response)
    validated = Person.model_validate(data)
except json.JSONDecodeError:
    # Handle invalid JSON
except ValidationError:
    # Handle schema mismatch

Best Practices

• Use JSON mode or structured outputs when available
• Provide example outputs in prompt
• Validate all outputs
• Handle partial/malformed responses
• Consider retry logic for failures