blue green
**Deployment Strategies for ML Models**
**Deployment Strategies**
**Blue-Green Deployment**
Two identical environments, switch traffic instantly:
```
[Load Balancer]
|
+-----------+-----------+
| |
[Blue (current)] [Green (new)]
| |
active preparing
```
```bash
# Blue active, deploy to Green
kubectl apply -f green-deployment.yaml
# Verify Green is healthy
kubectl wait --for=condition=ready pod -l app=green
# Switch traffic
kubectl patch service llm-service -p '{"spec":{"selector":{"version":"green"}}}'
```
**Canary Deployment**
Gradual traffic shift:
```yaml
# Nginx Ingress Canary
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: llm-canary
annotations:
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "10" # 10% to canary
spec:
rules:
- host: api.example.com
http:
paths:
- path: /v1/completions
backend:
service:
name: llm-canary
port: 80
```
**A/B Testing**
Route by user attributes:
```python
def route_request(request, user_id):
# Hash user to consistent bucket
bucket = hash(user_id) % 100
if bucket < 10: # 10% to new model
return call_model_v2(request)
else:
return call_model_v1(request)
```
**ML Model Rollout**
```python
# Argo Rollouts example
apiVersion: argoproj.io/v1alpha1
kind: Rollout
spec:
strategy:
canary:
steps:
- setWeight: 5
- pause: {duration: 10m}
- setWeight: 25
- pause: {duration: 10m}
- setWeight: 50
- pause: {duration: 10m}
- setWeight: 100
analysis:
templates:
- templateName: success-rate
```
**Comparison**
| Strategy | Risk | Rollback | Resource Cost |
|----------|------|----------|---------------|
| Blue-Green | Low | Instant | 2x |
| Canary | Low | Fast | 1.1x |
| Rolling | Medium | Slow | 1x |
| Recreate | High | Slow | 1x |
**ML-Specific Concerns**
| Concern | Solution |
|---------|----------|
| Model warm-up | Startup probe, pre-warming |
| GPU memory | Limit concurrent versions |
| A/B metrics | Compare model quality |
| Consistency | Session affinity if needed |
**Best Practices**
- Always have rollback plan
- Monitor model quality metrics during rollout
- Use canary for high-risk changes
- Automate deployment pipeline