json mode
**Structured Output and JSON Mode**
**Why Structured Output?**
LLMs naturally produce free-form text. For programmatic use, we need reliable structured output (JSON, XML, etc.).
**OpenAI JSON Mode**
**Basic JSON Mode**
```python
response = client.chat.completions.create(
model="gpt-4o",
messages=[{
"role": "user",
"content": "Extract name and age from: John is 30 years old"
}],
response_format={"type": "json_object"}
)
data = json.loads(response.choices[0].message.content)
# {"name": "John", "age": 30}
```
**Structured Outputs with Schema**
```python
from pydantic import BaseModel
class Person(BaseModel):
name: str
age: int
occupation: str | None = None
response = client.beta.chat.completions.parse(
model="gpt-4o",
messages=[...],
response_format=Person
)
person = response.choices[0].message.parsed
print(person.name) # Typed access
```
**Instructor Library**
Popular library for structured outputs with any LLM:
```python
import instructor
from pydantic import BaseModel
client = instructor.from_openai(OpenAI())
class UserInfo(BaseModel):
name: str
age: int
email: str
user = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Extract: John, 30, [email protected]"}],
response_model=UserInfo
)
print(user.name) # "John"
```
**Outlines (for local models)**
Constrained generation ensuring valid JSON:
```python
from outlines import models, generate
model = models.transformers("meta-llama/Llama-2-7b-hf")
schema = {
"type": "object",
"properties": {
"name": {"type": "string"},
"age": {"type": "integer"}
},
"required": ["name", "age"]
}
generator = generate.json(model, schema)
result = generator("Extract from: John is 30")
```
**Validation**
Always validate LLM JSON output:
```python
from pydantic import ValidationError
try:
data = json.loads(response)
validated = Person.model_validate(data)
except json.JSONDecodeError:
# Handle invalid JSON
except ValidationError:
# Handle schema mismatch
```
**Best Practices**
- Use JSON mode or structured outputs when available
- Provide example outputs in prompt
- Validate all outputs
- Handle partial/malformed responses
- Consider retry logic for failures