Online Learning

What is Online Learning? Learning from streaming data one sample (or mini-batch) at a time, updating the model incrementally rather than retraining from scratch.

Online vs Batch Learning

Online Learning Algorithms

Stochastic Gradient Descent

def online_sgd(model, data_stream, lr=0.01):
    for sample in data_stream:
        x, y = sample
        prediction = model(x)
        loss = criterion(prediction, y)

        loss.backward()
        for param in model.parameters():
            param.data -= lr * param.grad
            param.grad.zero_()

Online Gradient Descent with Regret

# Track cumulative regret
cumulative_loss = 0
best_fixed_loss = compute_best_in_hindsight(data_stream)

for t, sample in enumerate(data_stream):
    loss = model.loss(sample)
    cumulative_loss += loss
    model.update(sample)

regret = cumulative_loss - best_fixed_loss
# Want sublinear regret: O(sqrt(T)) or O(log T)

Challenges

Concept Drift Detection

class DriftDetector:
    def __init__(self, window_size=100, threshold=0.05):
        self.window = deque(maxlen=window_size)
        self.threshold = threshold

    def update(self, error):
        self.window.append(error)

        if len(self.window) == self.window.maxlen:
            recent = list(self.window)[-50:]
            old = list(self.window)[:50]

            if mean(recent) - mean(old) > self.threshold:
                return True  # Drift detected
        return False

Use Cases

Frameworks

Best Practices

• Use appropriate learning rate schedules
• Monitor for concept drift
• Consider data buffering for stability
• Evaluate on recent data