Continual Learning Lifelong Learning Catastrophic Forgetting
# Continual Learning: Lifelong Learning & Catastrophic Forgetting
## Introduction & Motivation
Continual learning: learn sequentially from tasks; avoid forgetting previous. Catastrophic forgetting: learning new → forget old. Experience replay: maintain buffer; interleave old data. Elastic weight consolidation (EWC): protect important weights. Parameters isolation: task-specific parameters. Applications: lifelong learning, continual adaptation.
Motivation: Sequential tasks; limited storage. Balance learning new and retaining old.
Applications: Lifelong learning, continual adaptation, few-shot continual.
---
## Core Concepts & Theory
### Catastrophic Forgetting
Learning new task overwrites old knowledge.
### Replay
Store old data; interleave with new.
### Regularization
Protect important parameters from change.
---
## Mathematical Formulation
EWC loss:
$$L = L_{ ext{new}} + \frac{\lambda}{2} \sum_i F_i (w_i - w^*_i)^2$$
where F_i = Fisher information diagonal (importance).
Replay:
$$L = L_{ ext{new}} + L_{ ext{old}}$$
jointly minimize on new and old tasks.
---
## Advanced Theory & Extensions
### Memory-Augmented Networks
External memory; retrieve past experiences.
### Dynamic Expansion
Add new parameters for new tasks.
### Orthogonal Weight Modification
Keep old subspace orthogonal to new.
---
## Computational Considerations
Replay: O(stored_samples) memory.
EWC: O(parameters) Fisher storage.
Expansion: O(new_params) parameter growth.
---
## Practical Implementation Strategies
### Buffer Management
Reservoir sampling; maintain balanced classes.
### Task Boundaries
Know task boundaries; easier setting.
### Rehearsal Ratio
Balance new vs replayed; typically 1:1.
---
## Benchmark Datasets & Evaluation
Split CIFAR: Sequential tasks; continual learning standard.
Permuted MNIST: Permuted pixels; increasing difficulty.
CORe50: Real-world continual; visual recognition.
---
## Key Challenges & Limitations
### Storage Cost
Replay needs buffer; memory limited.
### Boundary Knowledge
Don't know task boundaries; harder.
### Metric Ambiguity
Forward vs backward transfer; depends.
---
## Hyperparameter Tuning
Buffer size: 1-10% of data typical.
EWC λ: 0.4-1.0; balance old-new.
Replay ratio: 1:1 typical; tune empirically.
---
## Real-World Applications & Case Studies
Robot Learning: Learn new environments; retain old.
Streaming Data: Evolving distribution; continual.
Few-Shot Continual: Learn one example; retain knowledge.
---
## Integration with Other Methods
Continual + Meta-Learning → rapid continual.
Continual + Generative Models → replay via generation.
---
## Summary & Key Takeaways
Continual learning via experience replay and elastic weight consolidation enables sequential task learning while mitigating catastrophic forgetting through memory and parameter regularization.
Principles:
1. Catastrophic forgetting: new learning overwrites old.
2. Replay: store and interleave old experiences.
3. EWC: protect important parameters.
4. Task-specific: separate parameters per task.
5. Forward/backward transfer: measure continual benefit.
---
---
## Appendix: Practical Labs
### Lab 1: Replay Buffer
import numpy as np
from collections import deque
class ReplayBuffer:
def __init__(self, capacity=1000):
self.buffer = deque(maxlen=capacity)
def push(self, x, y):
self.buffer.append((x, y))
def sample(self, batch_size):
indices = np.random.choice(len(self.buffer), batch_size, replace=False)
xs, ys = zip(*[self.buffer[i] for i in indices])
return np.array(xs), np.array(ys)
# Test
buffer = ReplayBuffer(capacity=100)
for i in range(50):
buffer.push(np.random.randn(10), i % 5)
xs, ys = buffer.sample(10)
assert xs.shape == (10, 10), "Samples shape"
assert ys.shape == (10,), "Labels shape"
print("✓ Replay buffer working")
if __name__ == "__main__":
print("Lab 1: Buffer - PASSED")### Lab 2: Elastic Weight Consolidation
import torch
import numpy as np
def compute_fisher_information(model, X, y):
"""Estimate Fisher information diagonal"""
fisher = {}
for name, param in model.named_parameters():
fisher[name] = torch.zeros_like(param)
for x, label in zip(X, y):
output = model(x.unsqueeze(0))
loss = output[0, label]
loss.backward(retain_graph=True)
for name, param in model.named_parameters():
if param.grad is not None:
fisher[name] += param.grad ** 2
return fisher
# Test
np.random.seed(42)
model = torch.nn.Linear(10, 5)
X = torch.randn(20, 10)
y = torch.randint(0, 5, (20,))
fisher = compute_fisher_information(model, X, y)
assert len(fisher) > 0, "Fisher computed"
print("✓ Fisher information working")
if __name__ == "__main__":
print("Lab 2: Fisher - PASSED")### Lab 3: Task-Aware Learning
import numpy as np
def task_aware_training(model, X_old, y_old, X_new, y_new, old_task_id=0, new_task_id=1):
"""Train on new task; keep old task performance"""
# Train on both old and new
losses = []
for x, y in list(zip(X_old, y_old)) + list(zip(X_new, y_new)):
loss = ((x ** 2).sum() - y) ** 2 # Dummy loss
losses.append(loss)
avg_loss = np.mean(losses)
return avg_loss
# Test
np.random.seed(42)
X_old = np.random.randn(20, 10)
y_old = np.random.randn(20)
X_new = np.random.randn(20, 10)
y_new = np.random.randn(20)
loss = task_aware_training(None, X_old, y_old, X_new, y_new)
assert np.isfinite(loss), "Loss finite"
print("✓ Task-aware training working")
if __name__ == "__main__":
print("Lab 3: TaskAware - PASSED")### Lab 4: Continual Learning Evaluation
import numpy as np
def compute_forward_transfer(task_accs):
"""Forward transfer: performance on new tasks"""
# Average accuracy on new tasks
forward = np.mean(task_accs)
return forward
def compute_backward_transfer(task_accs_before, task_accs_after):
"""Backward transfer: retention of old task performance"""
# Difference in old task accuracy
backward = np.mean(task_accs_after - task_accs_before)
return backward
# Test
np.random.seed(42)
accs_new = [0.8, 0.75, 0.7] # Accuracy on new tasks
accs_old_before = [0.9, 0.92] # Old task accuracy before
accs_old_after = [0.75, 0.7] # Old task accuracy after
forward = compute_forward_transfer(accs_new)
backward = compute_backward_transfer(accs_old_before, accs_old_after)
assert 0 <= forward <= 1, "Forward transfer in [0,1]"
assert backward < 0, "Backward transfer negative (forgetting)"
print("✓ Continual evaluation working")
if __name__ == "__main__":
print("Lab 4: Evaluation - PASSED")