circuit breaker pattern
**Circuit Breaker Pattern** is the **software resilience pattern that prevents cascading failures by automatically stopping requests to failing services** — allowing downstream services time to recover while protecting calling services from thread pool exhaustion, timeout accumulation, and resource depletion that would otherwise propagate a single service failure into a system-wide outage across an entire microservices architecture.
**What Is the Circuit Breaker Pattern?**
- **Definition**: A design pattern that wraps calls to external services in a stateful proxy that monitors failures and automatically short-circuits requests when a failure threshold is exceeded.
- **Analogy**: Works like an electrical circuit breaker — when current (failures) exceeds safe limits, the breaker trips to protect the system.
- **Origin**: Popularized by Michael Nygard in "Release It!" and implemented in Netflix's Hystrix library.
- **Core Value**: Provides fail-fast behavior that preserves system resources and enables automatic recovery detection.
**The Three States**
- **Closed (Normal Operation)**: Requests pass through to the downstream service normally. The breaker monitors failure rates and counts consecutive failures.
- **Open (Service Failing)**: After failures exceed the threshold, the breaker opens. All requests fail immediately without calling the downstream service, returning a fallback response or error instantly.
- **Half-Open (Testing Recovery)**: After a configured timeout period, the breaker allows a limited number of test requests through. If tests succeed, the breaker closes (recovery confirmed). If tests fail, it reopens.
**Why Circuit Breakers Matter**
- **Prevent Cascading Failures**: One failing service can exhaust connection pools and threads in every service that calls it, cascading across the system.
- **Reduce Latency During Failures**: Instead of waiting 30 seconds for a timeout, requests fail in milliseconds when the breaker is open.
- **Protect Resources**: Thread pools, database connections, and memory are preserved for healthy request paths.
- **Enable Graceful Degradation**: Open breakers trigger fallback logic that provides reduced but functional service.
- **Automatic Recovery**: The half-open state automatically detects when failed services recover, restoring normal operation without manual intervention.
**Configuration Parameters**
| Parameter | Description | Typical Value |
|-----------|-------------|---------------|
| **Failure Threshold** | Number or percentage of failures to trip breaker | 5 consecutive or 50% in window |
| **Timeout Period** | How long breaker stays open before testing | 30-60 seconds |
| **Half-Open Limit** | Number of test requests in half-open state | 1-3 requests |
| **Monitoring Window** | Time window for counting failures | 10-60 seconds |
| **Success Threshold** | Successes needed in half-open to close | 3-5 consecutive |
**Implementation in ML Systems**
- **Model Serving**: Breakers on inference endpoints prevent one slow model from blocking request threads.
- **Feature Stores**: Breakers on feature retrieval trigger cached or default feature fallbacks.
- **External APIs**: Breakers on third-party API calls (enrichment, validation) protect core prediction paths.
- **Data Pipelines**: Breakers on upstream data sources prevent pipeline stalls from propagating downstream.
**Popular Implementations**
- **Resilience4j**: Modern Java circuit breaker library replacing Netflix Hystrix.
- **Polly (.NET)**: Circuit breaker and resilience library for .NET applications.
- **Istio/Envoy**: Service mesh-level circuit breaking without application code changes.
- **Python**: tenacity, pybreaker, and custom implementations with decorators.
Circuit Breaker Pattern is **the essential resilience primitive for distributed systems** — providing automatic failure isolation and recovery detection that prevents individual service failures from cascading into system-wide outages, making it a mandatory component of any production microservices architecture.