audit log

**Audit logging for LLMs** Audit logging for LLMs is a critical compliance and security requirement that captures detailed records of all system interactions to ensure accountability, traceability, and regulatory adherence. Data to capture: full inputs (prompts), full outputs (responses), timestamps, user definition, model version, and hyperparameters. Sensitive data: logs often contain PII or confidential IP; must be encrypted and access-controlled. Compliance standards: SOC2, HIPAA, and GDPR often require audit trails for data access and processing. Anomaly detection: analyze logs for abuse patterns (prompt injection attempts, high-volume scraping). Debugging: essential for tracing quality issues or hallucinations reported by users. Retention policy: define how long logs are kept (e.g., 90 days hot storage, 1 year cold); balances cost vs. compliance. Non-repudiation: logs provide evidence of what system actually generated. Implementation: middleware or gateway layer (like LiteLLM or custom proxy) best place to capture traffic. Redaction: automatic PII redaction before logging may be necessary for some privacy standards. Audit logging transforms LLM interactions from ephemeral events into a verifiable record of operations.