High availability (HA) is the design of a system to ensure it remains operational and accessible for a very high percentage of time, minimizing downtime even during hardware failures, software bugs, network issues, or maintenance activities.

Availability Levels (The "Nines")

• 99% (two nines): ~87.6 hours downtime/year — unacceptable for most services.
• 99.9% (three nines): ~8.76 hours downtime/year — acceptable for internal tools.
• 99.95%: ~4.38 hours downtime/year — common SLA target for cloud services.
• 99.99% (four nines): ~52.6 minutes downtime/year — high availability standard.
• 99.999% (five nines): ~5.26 minutes downtime/year — carrier-grade availability.

HA Architecture Patterns

• Redundancy: Run multiple instances of every component — if one fails, others continue serving.
• Load Balancing: Distribute traffic across instances. Healthy instances absorb traffic from failed ones.
• Active-Active: Multiple instances actively serving traffic simultaneously. Highest availability but most complex.
• Active-Passive: One instance serves traffic; a standby takes over on failure (failover). Simpler but slower recovery.
• Multi-Region: Deploy in multiple geographic regions so a regional outage doesn't cause global downtime.

HA for AI/ML Systems

• Multi-Model Redundancy: If the primary LLM API (OpenAI) is down, automatically route to a backup (Anthropic, self-hosted).
• GPU Redundancy: Maintain spare GPU capacity or use multiple GPU providers.
• Database Replication: Replicate vector databases and application databases across zones or regions.
• Stateless Services: Design inference services to be stateless — any instance can handle any request, making failover instant.

HA Challenges for AI

• GPU Scarcity: GPU instances are expensive and often capacity-constrained — maintaining hot standby GPUs is costly.
• Model Loading Time: Large models take minutes to load onto GPUs, creating cold-start delays during failover.
• State Management: KV cache and session state must be handled carefully to avoid losing context during failover.

Calculating System Availability

For components in series: $A_{total} = A_1 \times A_2 \times A_3$ For redundant components: $A_{total} = 1 - (1 - A_1)(1 - A_2)$

High availability is achieved through redundancy at every layer — no single component failure should take down the system.