Continual Learning Catastrophic Forgetting
# Continual Learning & Catastrophic Forgetting
## Introduction & Motivation
Continual Learning: learn from sequential tasks without forgetting previous knowledge. Catastrophic forgetting; lifelong learning. Applications: robotics, adaptive systems, evolving environments.
Motivation: Learn new tasks while retaining old knowledge.
Applications: Continual task adaptation, lifelong learning systems.
---
## Core Concepts & Theory
### Catastrophic Forgetting
Performance degradation on previous tasks.
### Elastic Weight Consolidation (EWC)
Regularize important weights.
### Replay Buffers
Store and replay old samples.
### Task-Specific Parameters
Maintain separate parameter groups.
---
## Mathematical Formulation
EWC Loss:
$$L = L_ ext{new} + \frac{\lambda}{2} \sum_i F_i (w_i - w^*_i)^2$$
Fisher Information Diagonal:
$$F_i = \mathbb{E}[(
abla_i L)^2]$$
Experience Replay:
$$L_ ext{total} = \alpha L_ ext{new} + (1-\alpha) L_ ext{replay}$$
---
## Advanced Theory & Extensions
### Packnet
Progressive masking for task allocation.
### DER (Dark Experience Replay)
Replay with compressed old data.
### Adapter Networks
Task-specific adaptation modules.
---
## Computational Considerations
Fisher computation: O(N·d²).
Replay storage: O(buffer_size·d).
Task-specific params: O(num_tasks·d).
---
## Practical Implementation Strategies
### Exemplar Selection
Sample representative old data.
### Task Boundaries
Clear separation between tasks.
### Plasticity-Stability Trade-off
Balance learning and retention.
---
## Benchmark Datasets & Evaluation
Split CIFAR-100: 100 sequential tasks.
Permuted MNIST: 10 permutation tasks.
CORe50: Real-world robot learning.
---
## Key Challenges & Limitations
### Negative Transfer
Old knowledge hurting new tasks.
### Computational Overhead
Storing and replaying data.
### Task Similarity
Performance depends on task order.
---
## Hyperparameter Tuning
EWC weight (λ): 0.01-1.0.
Replay ratio (1-α): 0.1-0.5.
Fisher damping: 0.01-0.1.
---
## Real-World Applications & Case Studies
Robotics: Sequential task learning.
Autonomous Systems: Adaptive to environment changes.
Recommendation Systems: Evolving user preferences.
---
## Integration with Other Methods
Continual learning + meta-learning for fast adaptation; + curriculum learning for task ordering.
---
## Summary & Key Takeaways
Continual Learning via EWC and replay buffers enables learning sequential tasks without catastrophic forgetting.
Principles:
1. Catastrophic forgetting: Main challenge.
2. Fisher regularization: Protect important weights.
3. Experience replay: Retain old knowledge.
4. Task-specific parameters: Modular learning.
5. Plasticity-stability: Balance trade-off.
---
---
## Appendix: Practical Labs
### Lab 1: Fisher Information
import numpy as np
def compute_fisher_information(gradients):
"""Compute Fisher information diagonal"""
# Fisher = E[(grad L)^2]
fisher = np.mean(gradients ** 2, axis=0)
return fisher
# Test
np.random.seed(42)
grads = np.random.randn(100, 50)
fisher = compute_fisher_information(grads)
assert fisher.shape == (50,), "Fisher shape"
assert np.all(fisher >= 0), "Fisher non-negative"
print("✓ Fisher information working")
if __name__ == "__main__":
print("Lab 1: FisherInformation - PASSED")### Lab 2: EWC Loss
import numpy as np
def ewc_loss(new_loss, weights, old_weights, fisher, lambda_ewc=0.4):
"""Elastic Weight Consolidation loss"""
# Quadratic penalty on weight changes
weight_diff = weights - old_weights
ewc_penalty = lambda_ewc / 2 * np.sum(fisher * (weight_diff ** 2))
total_loss = new_loss + ewc_penalty
return total_loss
# Test
new_loss = 0.5
weights = np.random.randn(50)
old_weights = weights + np.random.randn(50) * 0.1
fisher = np.abs(np.random.randn(50))
loss = ewc_loss(new_loss, weights, old_weights, fisher)
assert np.isfinite(loss), "Loss finite"
assert loss >= new_loss, "EWC increases loss"
print("✓ EWC loss working")
if __name__ == "__main__":
print("Lab 2: EWCLoss - PASSED")### Lab 3: Experience Replay Buffer
import numpy as np
class ReplayBuffer:
def __init__(self, max_size=1000):
self.max_size = max_size
self.buffer = []
def add(self, sample):
"""Add sample to buffer"""
if len(self.buffer) < self.max_size:
self.buffer.append(sample)
else:
# Replace oldest
self.buffer.pop(0)
self.buffer.append(sample)
def sample(self, batch_size):
"""Sample random batch"""
indices = np.random.choice(len(self.buffer), batch_size, replace=False)
return [self.buffer[i] for i in indices]
# Test
buffer = ReplayBuffer(max_size=10)
for i in range(15):
buffer.add(i)
assert len(buffer.buffer) == 10, "Buffer size capped"
batch = buffer.sample(5)
assert len(batch) == 5, "Batch size correct"
print("✓ Replay buffer working")
if __name__ == "__main__":
print("Lab 3: ReplayBuffer - PASSED")### Lab 4: Task-Specific Parameters
import numpy as np
class TaskSpecificModel:
def __init__(self, input_dim, hidden_dim):
self.input_dim = input_dim
self.hidden_dim = hidden_dim
self.shared_weights = np.random.randn(input_dim, hidden_dim) * 0.01
self.task_heads = {}
def add_task(self, task_id):
"""Add new task head"""
self.task_heads[task_id] = np.random.randn(self.hidden_dim, 10) * 0.01
def forward(self, x, task_id):
"""Forward pass for specific task"""
hidden = x @ self.shared_weights
output = hidden @ self.task_heads[task_id]
return output
# Test
model = TaskSpecificModel(input_dim=50, hidden_dim=100)
model.add_task(0)
model.add_task(1)
x = np.random.randn(4, 50)
out0 = model.forward(x, 0)
out1 = model.forward(x, 1)
assert out0.shape == (4, 10), "Output shape"
assert not np.allclose(out0, out1), "Different outputs per task"
print("✓ Task-specific parameters working")
if __name__ == "__main__":
print("Lab 4: TaskSpecificParameters - PASSED")