Structured Logging

Structured Logging is the practice of emitting log records as machine-parseable structured data (typically JSON) rather than unstructured human-readable text — enabling powerful querying, aggregation, alerting, and analysis of AI system behavior, performance, and errors using SQL-like queries and dashboards rather than brittle string parsing and grep-based log hunting.

What Is Structured Logging?

• Definition: A logging approach where each log entry is a structured data object with defined fields (timestamp, level, message, request_id, user_id, model, latency_ms, token_count) rather than a free-form text string — making log data queryable like a database table.
• Contrast with Unstructured Logging:
• Unstructured: `[INFO 2024-01-15 10:32:15] Model predicted 'cat' with 0.92 confidence in 145ms`
• Structured: `{"timestamp": "2024-01-15T10:32:15Z", "level": "INFO", "event": "prediction", "class": "cat", "confidence": 0.92, "latency_ms": 145, "model_version": "v4.2", "request_id": "req_abc123"}`
• Queryable: Structured logs can be queried with SQL-like syntax — SELECT AVG(latency_ms) WHERE model_version = 'v4.2' AND confidence > 0.9 — impossible with unstructured text.
• Industry Standard: Modern observability platforms (Datadog, Splunk, Elasticsearch, CloudWatch Logs Insights) natively query structured JSON logs.

Why Structured Logging Matters for AI Systems

• Performance Analysis: Query `AVG(llm_latency_ms) GROUP BY model_name` to compare model performance across versions — impossible without structured fields.
• Error Diagnosis: Filter `WHERE error_type = 'rate_limit' AND retry_count > 3` to identify systematic retry failures — requires structured error fields.
• Cost Monitoring: Aggregate `SUM(input_tokens + output_tokens) GROUP BY user_id, DATE` for per-user token cost accounting — requires token count fields in every log.
• Hallucination Tracking: Log fact-check results structurally — `{"event": "fact_check", "result": "failed", "claim": "...", "source_contradiction": "..."}` — then query failure rates over time.
• Alerting: Alert on error_rate > 0.05 WHERE model = 'gpt-4o' or P95_latency > 5000 — requires numeric fields in structured log data.
• Audit Compliance: Reconstruct complete request histories for compliance audits by querying structured logs filtered by user_id, request_id, or date range.

Structured Logging Implementation

Python with structlog (Recommended):

import structlog
from datetime import datetime

logger = structlog.get_logger()

def process_llm_request(request_id: str, user_id: str, query: str) -> str:
    start_time = datetime.utcnow()

    try:
        response = llm.generate(query)
        duration_ms = (datetime.utcnow() - start_time).total_seconds() * 1000

        logger.info(
            "llm_request_completed",
            request_id=request_id,
            user_id=user_id,
            model="gpt-4o",
            input_tokens=count_tokens(query),
            output_tokens=count_tokens(response),
            latency_ms=round(duration_ms),
            success=True
        )
        return response

    except RateLimitError as e:
        logger.warning(
            "llm_rate_limit",
            request_id=request_id,
            user_id=user_id,
            retry_after=e.retry_after,
            success=False
        )
        raise

Output JSON:

{
  "timestamp": "2024-01-15T10:32:15.234Z",
  "level": "info",
  "event": "llm_request_completed",
  "request_id": "req_abc123",
  "user_id": "usr_456",
  "model": "gpt-4o",
  "input_tokens": 342,
  "output_tokens": 187,
  "latency_ms": 1847,
  "success": true
}

Key Fields for AI System Logs

Log Level Strategy for AI Systems

• DEBUG: Full prompts and responses (development only — high volume, PII risk).
• INFO: Request completion with token counts, latency, model version.
• WARNING: Retries, rate limits, format corrections, low-confidence outputs.
• ERROR: Failed requests after max retries, validation failures, unexpected exceptions.
• CRITICAL: Service-wide failures, circuit breaker trips, data loss events.

Structured Log Querying Examples

In CloudWatch Logs Insights:

-- Average latency by model
fields @timestamp, model, latency_ms
| filter event = "llm_request_completed"
| stats avg(latency_ms) as avg_latency by model
| sort avg_latency desc

-- Error rate by hour
filter success = 0
| stats count() as errors by bin(1h)

-- Token cost by user (top 10)
filter event = "llm_request_completed"
| stats sum(input_tokens + output_tokens) as total_tokens by user_id
| sort total_tokens desc
| limit 10

PII Handling in Logs

AI system logs must handle personally identifiable information carefully:
- Never log raw user query content in production without PII scrubbing.
- Log query metadata (length, topic classification) rather than content.
- Apply field-level encryption or masking for sensitive structured fields.
- Ensure log retention policies comply with GDPR, CCPA data deletion requirements.
- Use separate log streams for high-sensitivity data with stricter access controls.

Structured logging is the observability foundation that transforms AI systems from black boxes into monitorable, debuggable, and auditable production infrastructure — by emitting machine-parseable structured data from every significant operation, teams gain the ability to answer operational questions — why did that request fail, which model version is slower, which users are approaching token limits — with queries rather than grep, enabling data-driven AI operations at scale.