online learning
**Online Learning**
**What is Online Learning?**
Learning from streaming data one sample (or mini-batch) at a time, updating the model incrementally rather than retraining from scratch.
**Online vs Batch Learning**
| Aspect | Batch | Online |
|--------|-------|--------|
| Data access | Full dataset | One sample at a time |
| Training | Multiple epochs | Single pass |
| Memory | Store all data | Constant memory |
| Adaptation | Periodic retraining | Continuous updates |
**Online Learning Algorithms**
**Stochastic Gradient Descent**
```python
def online_sgd(model, data_stream, lr=0.01):
for sample in data_stream:
x, y = sample
prediction = model(x)
loss = criterion(prediction, y)
loss.backward()
for param in model.parameters():
param.data -= lr * param.grad
param.grad.zero_()
```
**Online Gradient Descent with Regret**
```python
# Track cumulative regret
cumulative_loss = 0
best_fixed_loss = compute_best_in_hindsight(data_stream)
for t, sample in enumerate(data_stream):
loss = model.loss(sample)
cumulative_loss += loss
model.update(sample)
regret = cumulative_loss - best_fixed_loss
# Want sublinear regret: O(sqrt(T)) or O(log T)
```
**Challenges**
| Challenge | Mitigation |
|-----------|------------|
| Concept drift | Adaptive learning rates, windowing |
| Catastrophic forgetting | Experience replay |
| Noisy samples | Robust loss functions |
| Non-stationarity | Discount old data |
**Concept Drift Detection**
```python
class DriftDetector:
def __init__(self, window_size=100, threshold=0.05):
self.window = deque(maxlen=window_size)
self.threshold = threshold
def update(self, error):
self.window.append(error)
if len(self.window) == self.window.maxlen:
recent = list(self.window)[-50:]
old = list(self.window)[:50]
if mean(recent) - mean(old) > self.threshold:
return True # Drift detected
return False
```
**Use Cases**
| Use Case | Examples |
|----------|----------|
| Recommendations | User preferences evolve |
| Fraud detection | Attack patterns change |
| NLP | Language trends shift |
| Finance | Market conditions change |
**Frameworks**
| Framework | Features |
|-----------|----------|
| River | Python online learning |
| Vowpal Wabbit | Fast online learning |
| Flink ML | Streaming ML |
**Best Practices**
- Use appropriate learning rate schedules
- Monitor for concept drift
- Consider data buffering for stability
- Evaluate on recent data