cortex
**Cortex: Serverless ML Infrastructure**
**Overview**
Cortex is an open-source platform for deploying machine learning models as production-ready web APIs. It automates the infrastructure underlying model serving on AWS (EC2, EKS), abstracting away Kubernetes and Docker complexity.
**Key Features**
**1. Unified Config**
Deploy TensorFlow, PyTorch, Scikit-learn, or ONNX models using a simple `cortex.yaml` file.
**2. Autoscaling**
Automatically scales the number of replicas based on request traffic (Requests Per Second) or GPU utilization. Scales to zero to save costs.
**3. Spot Instances**
Built-in support for AWS Spot Instances, potentially saving 70-90% on compute costs, with auto-recovery if instances are reclaimed.
**4. Rolling Updates**
Updates APIs without downtime.
**Configuration Example**
```yaml
# cortex.yaml
- name: sentiment-analyzer
kind: RealtimeAPI
predictor:
type: python
path: predictor.py
compute:
cpu: 1
gpu: 1 # Uses GPU instance
mem: 4G
autoscaling:
min_replicas: 1
max_replicas: 10
```
**Python Predictor**
```python
# predictor.py
class PythonPredictor:
def __init__(self, config):
self.model = load_model()
def predict(self, payload):
return self.model.inference(payload["text"])
```
**Status**
**Note**: Cortex was acquired by Databricks, and the open-source project is no longer actively maintained as of late 2021/2022.
Modern Alternatives include:
- **BentoML**: For packaging models.
- **Ray Serve**: For scalable serving.
- **KServe**: For Kubernetes native serving.
- **AWS SageMaker**: Managed alternative.