availability high

**High availability (HA)** is the design of a system to ensure it remains **operational and accessible** for a very high percentage of time, minimizing downtime even during hardware failures, software bugs, network issues, or maintenance activities. **Availability Levels (The "Nines")** - **99% (two nines)**: ~87.6 hours downtime/year — unacceptable for most services. - **99.9% (three nines)**: ~8.76 hours downtime/year — acceptable for internal tools. - **99.95%**: ~4.38 hours downtime/year — common SLA target for cloud services. - **99.99% (four nines)**: ~52.6 minutes downtime/year — high availability standard. - **99.999% (five nines)**: ~5.26 minutes downtime/year — carrier-grade availability. **HA Architecture Patterns** - **Redundancy**: Run multiple instances of every component — if one fails, others continue serving. - **Load Balancing**: Distribute traffic across instances. Healthy instances absorb traffic from failed ones. - **Active-Active**: Multiple instances actively serving traffic simultaneously. Highest availability but most complex. - **Active-Passive**: One instance serves traffic; a standby takes over on failure (failover). Simpler but slower recovery. - **Multi-Region**: Deploy in multiple geographic regions so a regional outage doesn't cause global downtime. **HA for AI/ML Systems** - **Multi-Model Redundancy**: If the primary LLM API (OpenAI) is down, automatically route to a backup (Anthropic, self-hosted). - **GPU Redundancy**: Maintain spare GPU capacity or use multiple GPU providers. - **Database Replication**: Replicate vector databases and application databases across zones or regions. - **Stateless Services**: Design inference services to be stateless — any instance can handle any request, making failover instant. **HA Challenges for AI** - **GPU Scarcity**: GPU instances are expensive and often capacity-constrained — maintaining hot standby GPUs is costly. - **Model Loading Time**: Large models take minutes to load onto GPUs, creating cold-start delays during failover. - **State Management**: KV cache and session state must be handled carefully to avoid losing context during failover. **Calculating System Availability** For components in series: $A_{total} = A_1 \times A_2 \times A_3$ For redundant components: $A_{total} = 1 - (1 - A_1)(1 - A_2)$ High availability is achieved through **redundancy at every layer** — no single component failure should take down the system.