SLO (Service Level Objective) is the specific, measurable reliability target that defines acceptable service performance for AI systems — the internal engineering goal that sits between the raw measurement (SLI) and the contractual obligation (SLA), giving teams a precise target to build toward and an error budget to spend on innovation vs stability.

What Is an SLO?

• Definition: A quantitative target for service reliability expressed as: "Metric X must achieve value Y for Z% of the time over rolling period P."
• The Three Terms:
• SLI (Service Level Indicator): The actual measured value — "Current p99 latency is 312ms."
• SLO (Service Level Objective): The engineering target — "p99 latency must be < 500ms for 99.5% of requests."
• SLA (Service Level Agreement): The legal contract — "If p99 latency exceeds 2s for > 0.5% of requests in a month, customers receive a 10% credit."
• Internal vs External: SLOs are internal engineering goals; SLAs are customer-facing contracts. SLOs are typically more aggressive than SLAs — if you only meet your SLO, you have buffer before violating the SLA.

Why SLOs Matter for AI Systems

• Quantified Reliability: "The model is slow" is unmeasurable. "p99 TTFT exceeds 3s for 0.2% of requests" is actionable — triggers an alert, consumes error budget, and demands a fix.
• Prioritization: SLOs answer "Is this worth fixing tonight?" — if you're well within SLO, the bug can wait. If you're burning error budget rapidly, it's an emergency.
• Innovation vs Reliability Balance: Error budgets derived from SLOs give teams permission to take risks (deploy new model versions, refactor serving infrastructure) when reliability is healthy.
• Cross-Team Alignment: SLOs provide a shared language between engineering, product, and business — "We are at 99.8% vs 99.9% SLO" is clearer than "performance is okay."
• Dependency Management: When upstream services (OpenAI API, vector DB) fail to meet their SLOs, your composite SLO helps you quantify and attribute the impact.

SLO Types for AI/LLM Systems

Availability SLO:

• "The inference API must return a non-5xx response for >= 99.9% of requests over any 30-day window."
• Measured as: successful_requests / total_requests.

Latency SLO:

• "Time to First Token (TTFT) must be < 2 seconds for >= 95% of requests."
• "End-to-end response time must be < 30 seconds for >= 99% of requests."
• Measured using histograms with Prometheus histogram_quantile().

Quality SLO:

• "Semantic similarity score vs golden answers must be >= 0.75 for >= 90% of evaluation set queries."
• "Retrieval precision@5 must be >= 0.8 on weekly evaluation runs."

Cost SLO:

• "Average cost per query must not exceed $0.05 over any 7-day window."
• Prevents runaway costs from prompt injection or misconfigured clients.

Throughput SLO:

• "System must sustain >= 100 concurrent users with < 5% error rate."
• "Token generation throughput must be >= 50 tokens/second per GPU."

SLO Design Guidelines

• Start with users: What latency do users actually notice? Research shows users perceive > 200ms delays — set SLO tighter than user perception threshold.
• Use percentiles, not averages: Average hides tail latency. p99 at 10s means 1 in 100 requests is terrible — use p95/p99/p99.9.
• Rolling windows: 30-day rolling windows are standard — they capture recent trends without overly punishing isolated incidents.
• Don't target 100%: 100% SLO is unachievable and incentivizes avoiding all change. 99.9% is "three nines" — 43 minutes of allowed downtime per month.

SLO Examples for Common AI Services

Error Budget = 100% - SLO Target

At 99.9% SLO over 30 days: 30 × 24 × 60 × 0.001 = 43.2 minutes of allowed downtime. When error budget is healthy (> 50% remaining): teams can safely deploy new model versions, run experiments. When error budget is depleted (< 10% remaining): freeze risky changes, focus on reliability improvements.

SLOs are the foundation of data-driven reliability engineering for AI systems — by making reliability targets explicit, measurable, and tied to user experience, SLOs transform vague aspirations like "the system should be fast and reliable" into precise engineering goals with clear accountability and the ability to make rational trade-offs between innovation velocity and production stability.