Structured Logging is the practice of emitting log records as machine-parseable structured data (typically JSON) rather than unstructured human-readable text — enabling powerful querying, aggregation, alerting, and analysis of AI system behavior, performance, and errors using SQL-like queries and dashboards rather than brittle string parsing and grep-based log hunting.
What Is Structured Logging?
- Definition: A logging approach where each log entry is a structured data object with defined fields (timestamp, level, message, request_id, user_id, model, latency_ms, token_count) rather than a free-form text string — making log data queryable like a database table.
- Contrast with Unstructured Logging:
- Unstructured: [INFO 2024-01-15 10:32:15] Model predicted 'cat' with 0.92 confidence in 145ms
- Structured: {"timestamp": "2024-01-15T10:32:15Z", "level": "INFO", "event": "prediction", "class": "cat", "confidence": 0.92, "latency_ms": 145, "model_version": "v4.2", "request_id": "req_abc123"}
- Queryable: Structured logs can be queried with SQL-like syntax — SELECT AVG(latency_ms) WHERE model_version = 'v4.2' AND confidence > 0.9 — impossible with unstructured text.
- Industry Standard: Modern observability platforms (Datadog, Splunk, Elasticsearch, CloudWatch Logs Insights) natively query structured JSON logs.
Why Structured Logging Matters for AI Systems
- Performance Analysis: Query AVG(llm_latency_ms) GROUP BY model_name to compare model performance across versions — impossible without structured fields.
- Error Diagnosis: Filter WHERE error_type = 'rate_limit' AND retry_count > 3 to identify systematic retry failures — requires structured error fields.
- Cost Monitoring: Aggregate SUM(input_tokens + output_tokens) GROUP BY user_id, DATE for per-user token cost accounting — requires token count fields in every log.
- Hallucination Tracking: Log fact-check results structurally — {"event": "fact_check", "result": "failed", "claim": "...", "source_contradiction": "..."} — then query failure rates over time.
- Alerting: Alert on error_rate > 0.05 WHERE model = 'gpt-4o' or P95_latency > 5000 — requires numeric fields in structured log data.
- Audit Compliance: Reconstruct complete request histories for compliance audits by querying structured logs filtered by user_id, request_id, or date range.
Structured Logging Implementation
Python with structlog (Recommended):
``python
import structlog
from datetime import datetime
logger = structlog.get_logger()
def process_llm_request(request_id: str, user_id: str, query: str) -> str:
start_time = datetime.utcnow()
try:
response = llm.generate(query)
duration_ms = (datetime.utcnow() - start_time).total_seconds() * 1000
logger.info(
"llm_request_completed",
request_id=request_id,
user_id=user_id,
model="gpt-4o",
input_tokens=count_tokens(query),
output_tokens=count_tokens(response),
latency_ms=round(duration_ms),
success=True
)
return response
except RateLimitError as e:
logger.warning(
"llm_rate_limit",
request_id=request_id,
user_id=user_id,
retry_after=e.retry_after,
success=False
)
raise
`
Output JSON:
`json`
{
"timestamp": "2024-01-15T10:32:15.234Z",
"level": "info",
"event": "llm_request_completed",
"request_id": "req_abc123",
"user_id": "usr_456",
"model": "gpt-4o",
"input_tokens": 342,
"output_tokens": 187,
"latency_ms": 1847,
"success": true
}
Key Fields for AI System Logs
| Field | Type | Purpose |
|-------|------|---------|
| timestamp | ISO 8601 | Time correlation |
| request_id | UUID | Request tracing |
| user_id | String | Per-user analysis |
| session_id | String | Conversation tracking |
| event | String | Log type classification |
| model | String | Model version tracking |
| input_tokens | Integer | Cost accounting |
| output_tokens | Integer | Cost accounting |
| latency_ms | Integer | Performance monitoring |
| retry_count | Integer | Reliability tracking |
| error_type | String | Error classification |
| rag_chunks_retrieved | Integer | RAG performance |
| confidence | Float | Quality tracking |
| success | Boolean | Success rate monitoring |
Log Level Strategy for AI Systems
- DEBUG: Full prompts and responses (development only — high volume, PII risk).
- INFO: Request completion with token counts, latency, model version.
- WARNING: Retries, rate limits, format corrections, low-confidence outputs.
- ERROR: Failed requests after max retries, validation failures, unexpected exceptions.
- CRITICAL: Service-wide failures, circuit breaker trips, data loss events.
Structured Log Querying Examples
In CloudWatch Logs Insights:
`sql
-- Average latency by model
fields @timestamp, model, latency_ms
| filter event = "llm_request_completed"
| stats avg(latency_ms) as avg_latency by model
| sort avg_latency desc
-- Error rate by hour
filter success = 0
| stats count() as errors by bin(1h)
-- Token cost by user (top 10)
filter event = "llm_request_completed"
| stats sum(input_tokens + output_tokens) as total_tokens by user_id
| sort total_tokens desc
| limit 10
``
PII Handling in Logs
AI system logs must handle personally identifiable information carefully:
- Never log raw user query content in production without PII scrubbing.
- Log query metadata (length, topic classification) rather than content.
- Apply field-level encryption or masking for sensitive structured fields.
- Ensure log retention policies comply with GDPR, CCPA data deletion requirements.
- Use separate log streams for high-sensitivity data with stricter access controls.
Structured logging is the observability foundation that transforms AI systems from black boxes into monitorable, debuggable, and auditable production infrastructure — by emitting machine-parseable structured data from every significant operation, teams gain the ability to answer operational questions — why did that request fail, which model version is slower, which users are approaching token limits — with queries rather than grep, enabling data-driven AI operations at scale.