container registry
**Container Registries for ML**
**Why Container Registries?**
Store and deploy ML model containers with versioning, security scanning, and access control.
**Major Registries**
| Registry | Provider | Features |
|----------|----------|----------|
| ECR | AWS | IAM integration, scanning |
| GCR/Artifact Registry | GCP | Multi-region, scanning |
| ACR | Azure | AAD integration |
| Docker Hub | Docker | Public images |
| Harbor | Self-hosted | Enterprise features |
**ECR Setup**
```bash
# Create repository
aws ecr create-repository --repository-name llm-inference
# Authenticate Docker
aws ecr get-login-password | docker login --username AWS --password-stdin
123456789.dkr.ecr.us-east-1.amazonaws.com
# Build and push
docker build -t llm-inference .
docker tag llm-inference:latest 123456789.dkr.ecr.us-east-1.amazonaws.com/llm-inference:v1
docker push 123456789.dkr.ecr.us-east-1.amazonaws.com/llm-inference:v1
```
**Image Tagging Strategy**
```bash
# Tag by version
llm-inference:1.0.0
llm-inference:1.0.1
# Tag by git commit
llm-inference:abc1234
# Tag by model version
llm-inference:gpt4-v2
# Tag by date
llm-inference:2024-01-15
```
**ML-Specific Considerations**
| Consideration | Solution |
|---------------|----------|
| Large images (10GB+) | Multi-stage builds, layer caching |
| Model weights | Separate from code, mount at runtime |
| GPU dependencies | Use NVIDIA base images |
| Security | Scan for vulnerabilities |
**Dockerfile for ML**
```dockerfile
# Multi-stage build
FROM python:3.11-slim as builder
COPY requirements.txt .
RUN pip wheel --no-cache-dir --wheel-dir=/wheels -r requirements.txt
FROM nvidia/cuda:12.1-runtime-ubuntu22.04
COPY --from=builder /wheels /wheels
RUN pip install --no-cache /wheels/*
COPY app/ /app/
WORKDIR /app
# Dont include model weights in image
# Mount from S3 or volume at runtime
ENTRYPOINT ["python", "serve.py"]
```
**Kubernetes ImagePullPolicy**
```yaml
spec:
containers:
- name: llm-server
image: 123456.dkr.ecr.us-east-1.amazonaws.com/llm-inference:v1.2.0
imagePullPolicy: IfNotPresent # Cache locally
```
**Best Practices**
- Use immutable tags (version, not :latest)
- Enable vulnerability scanning
- Clean up old images (lifecycle policies)
- Use multi-stage builds for smaller images
- Store model weights separately from code