Continual Learning Catastrophic Forgetting

# Continual Learning & Catastrophic Forgetting

## Introduction & Motivation

Continual Learning: learn from sequential tasks without forgetting previous knowledge. Catastrophic forgetting; lifelong learning. Applications: robotics, adaptive systems, evolving environments.

Motivation: Learn new tasks while retaining old knowledge.

Applications: Continual task adaptation, lifelong learning systems.

---

## Core Concepts & Theory

### Catastrophic Forgetting

Performance degradation on previous tasks.

### Elastic Weight Consolidation (EWC)

Regularize important weights.

### Replay Buffers

Store and replay old samples.

### Task-Specific Parameters

Maintain separate parameter groups.

---

## Mathematical Formulation

EWC Loss:
$$L = L_ ext{new} + \frac{\lambda}{2} \sum_i F_i (w_i - w^*_i)^2$$

Fisher Information Diagonal:
$$F_i = \mathbb{E}[( abla_i L)^2]$$

Experience Replay:
$$L_ ext{total} = \alpha L_ ext{new} + (1-\alpha) L_ ext{replay}$$

---

## Advanced Theory & Extensions

### Packnet

Progressive masking for task allocation.

### DER (Dark Experience Replay)

Replay with compressed old data.

### Adapter Networks

Task-specific adaptation modules.

---

## Computational Considerations

Fisher computation: O(N·d²).

Replay storage: O(buffer_size·d).

Task-specific params: O(num_tasks·d).

---

## Practical Implementation Strategies

### Exemplar Selection

Sample representative old data.

### Task Boundaries

Clear separation between tasks.

### Plasticity-Stability Trade-off

Balance learning and retention.

---

## Benchmark Datasets & Evaluation

Split CIFAR-100: 100 sequential tasks.

Permuted MNIST: 10 permutation tasks.

CORe50: Real-world robot learning.

---

## Key Challenges & Limitations

### Negative Transfer

Old knowledge hurting new tasks.

### Computational Overhead

Storing and replaying data.

### Task Similarity

Performance depends on task order.

---

## Hyperparameter Tuning

EWC weight (λ): 0.01-1.0.

Replay ratio (1-α): 0.1-0.5.

Fisher damping: 0.01-0.1.

---

## Real-World Applications & Case Studies

Robotics: Sequential task learning.

Autonomous Systems: Adaptive to environment changes.

Recommendation Systems: Evolving user preferences.

---

## Integration with Other Methods

Continual learning + meta-learning for fast adaptation; + curriculum learning for task ordering.

---

## Summary & Key Takeaways

Continual Learning via EWC and replay buffers enables learning sequential tasks without catastrophic forgetting.

Principles:
1. Catastrophic forgetting: Main challenge.
2. Fisher regularization: Protect important weights.
3. Experience replay: Retain old knowledge.
4. Task-specific parameters: Modular learning.
5. Plasticity-stability: Balance trade-off.

---

---

## Appendix: Practical Labs

### Lab 1: Fisher Information

import numpy as np

def compute_fisher_information(gradients):
 """Compute Fisher information diagonal"""
 # Fisher = E[(grad L)^2]
 fisher = np.mean(gradients ** 2, axis=0)
 
 return fisher

# Test
np.random.seed(42)
grads = np.random.randn(100, 50)

fisher = compute_fisher_information(grads)

assert fisher.shape == (50,), "Fisher shape"
assert np.all(fisher >= 0), "Fisher non-negative"
print("✓ Fisher information working")

if __name__ == "__main__":
 print("Lab 1: FisherInformation - PASSED")

### Lab 2: EWC Loss

import numpy as np

def ewc_loss(new_loss, weights, old_weights, fisher, lambda_ewc=0.4):
 """Elastic Weight Consolidation loss"""
 # Quadratic penalty on weight changes
 weight_diff = weights - old_weights
 ewc_penalty = lambda_ewc / 2 * np.sum(fisher * (weight_diff ** 2))
 
 total_loss = new_loss + ewc_penalty
 
 return total_loss

# Test
new_loss = 0.5
weights = np.random.randn(50)
old_weights = weights + np.random.randn(50) * 0.1
fisher = np.abs(np.random.randn(50))

loss = ewc_loss(new_loss, weights, old_weights, fisher)

assert np.isfinite(loss), "Loss finite"
assert loss >= new_loss, "EWC increases loss"
print("✓ EWC loss working")

if __name__ == "__main__":
 print("Lab 2: EWCLoss - PASSED")

### Lab 3: Experience Replay Buffer

import numpy as np

class ReplayBuffer:
 def __init__(self, max_size=1000):
 self.max_size = max_size
 self.buffer = []
 
 def add(self, sample):
 """Add sample to buffer"""
 if len(self.buffer) < self.max_size:
 self.buffer.append(sample)
 else:
 # Replace oldest
 self.buffer.pop(0)
 self.buffer.append(sample)
 
 def sample(self, batch_size):
 """Sample random batch"""
 indices = np.random.choice(len(self.buffer), batch_size, replace=False)
 return [self.buffer[i] for i in indices]

# Test
buffer = ReplayBuffer(max_size=10)

for i in range(15):
 buffer.add(i)

assert len(buffer.buffer) == 10, "Buffer size capped"
batch = buffer.sample(5)
assert len(batch) == 5, "Batch size correct"
print("✓ Replay buffer working")

if __name__ == "__main__":
 print("Lab 3: ReplayBuffer - PASSED")

### Lab 4: Task-Specific Parameters

import numpy as np

class TaskSpecificModel:
 def __init__(self, input_dim, hidden_dim):
 self.input_dim = input_dim
 self.hidden_dim = hidden_dim
 self.shared_weights = np.random.randn(input_dim, hidden_dim) * 0.01
 self.task_heads = {}
 
 def add_task(self, task_id):
 """Add new task head"""
 self.task_heads[task_id] = np.random.randn(self.hidden_dim, 10) * 0.01
 
 def forward(self, x, task_id):
 """Forward pass for specific task"""
 hidden = x @ self.shared_weights
 output = hidden @ self.task_heads[task_id]
 return output

# Test
model = TaskSpecificModel(input_dim=50, hidden_dim=100)
model.add_task(0)
model.add_task(1)

x = np.random.randn(4, 50)
out0 = model.forward(x, 0)
out1 = model.forward(x, 1)

assert out0.shape == (4, 10), "Output shape"
assert not np.allclose(out0, out1), "Different outputs per task"
print("✓ Task-specific parameters working")

if __name__ == "__main__":
 print("Lab 4: TaskSpecificParameters - PASSED")