online learning
**Online Learning and Concept Drift Adaptation** is the **machine learning paradigm where models are updated continuously as individual data points or small batches arrive in a stream** — contrasting with offline/batch learning where a fixed dataset is trained on once, enabling adaptation to non-stationary environments where the underlying data distribution changes over time (concept drift), as occurs in financial markets, user behavior, sensor networks, and evolving adversarial settings.
**Online Learning Fundamentals**
- **Regret minimization**: Online learning frames learning as a game against adversary.
- Cumulative regret: R_T = Σ ℓ(y_t, f(x_t)) - min_f Σ ℓ(y_t, f(x_t))
- Goal: Sub-linear regret R_T/T → 0 as T → ∞ (convergence to best fixed model).
- **Online gradient descent**: At each step t: w_{t+1} = w_t - η∇ℓ(y_t, f_w(x_t)).
- **Perceptron algorithm**: Mistake-driven; update only on misclassification.
**Types of Concept Drift**
- **Sudden drift**: Abrupt distribution change (e.g., marketing campaign changes user behavior).
- **Gradual drift**: Slow shift over time (e.g., seasonal patterns, aging sensors).
- **Recurring drift**: Cyclic patterns (e.g., weekday vs weekend behavior).
- **Incremental drift**: Gradual linear shift in decision boundary.
**Drift Detection Methods**
- **ADWIN (Adaptive Windowing)**: Maintains adaptive sliding window; triggers alarm when subwindows have significantly different means.
- Automatically adjusts window size → large window in stable periods, small after drift.
- **DDM (Drift Detection Method)**: Monitors classification error rate; raises warning/alarm when error significantly exceeds historical minimum.
- **KSWIN**: Kolmogorov-Smirnov test on sliding window → detects distribution shift in raw data.
- **Page-Hinkley test**: Sequential analysis; detects sustained increase in cumulative sum → gradual drift.
**Adaptive Algorithms**
- **ADWIN + classifier**: Replace classifier with retrained version when ADWIN triggers drift alarm.
- **Adaptive Random Forest (ARF)**: Ensemble of trees; each tree monitors its own drift detector; replaces drifted trees with new ones.
- **Hoeffding Trees**: Incrementally built decision trees using Hoeffding bound to determine when sufficient samples seen → no retraining.
- **Learn++**: Combines multiple classifiers trained on different time windows.
**Deep Learning Online Adaptation**
- **Elastic Weight Consolidation (EWC)**: Adds regularization term penalizing changes to weights important for previous tasks → prevents catastrophic forgetting during continual updates.
- **Experience replay**: Maintain small buffer of past examples → interleave with new samples → prevents forgetting.
- **Test-time adaptation (TTA)**: At inference, adapt BN statistics or model parameters to incoming batch without labels.
**Python: River ML Library**
```python
from river import linear_model, preprocessing, metrics, drift
# Online logistic regression with drift detection
model = linear_model.LogisticRegression()
scaler = preprocessing.StandardScaler()
detector = drift.ADWIN()
acc = metrics.Accuracy()
for x, y in data_stream:
x_scaled = scaler.learn_one(x).transform_one(x)
y_pred = model.predict_one(x_scaled)
model.learn_one(x_scaled, y) # incremental update
acc.update(y, y_pred)
detector.update(int(y_pred != y)) # track error rate
if detector.drift_detected:
model = linear_model.LogisticRegression() # reset model
```
**Applications**
- **Fraud detection**: Transaction patterns evolve as fraudsters adapt → must update in real time.
- **Recommendation systems**: User preferences change → online CF updates item/user embeddings.
- **Predictive maintenance**: Sensor drift → failure patterns change → online models adapt.
- **Network intrusion**: New attack patterns emerge → online classifiers retrain automatically.
Online learning and concept drift adaptation are **the temporal intelligence layer that keeps AI systems relevant in a changing world** — while offline models gradually degrade as the world they were trained on diverges from current reality, online learning systems continuously maintain accuracy by treating every new data point as a training signal, making them essential for any application where the cost of a stale model compounds over time, from trading algorithms that must adapt to market regime changes within minutes to fraud detectors that must recognize new attack patterns before significant losses accumulate.