continual learning
**Continual Learning**
**What is Continual Learning?**
Learning new tasks sequentially without forgetting previously learned tasks, enabling models to accumulate knowledge over time.
**The Forgetting Problem**
When training on new tasks, models tend to overwrite weights for old tasks:
```
Task 1: Learn A, B, C --> Model knows A, B, C
Task 2: Learn D, E --> Model knows D, E, forgets A, B, C
```
This is called "catastrophic forgetting."
**Approaches to Prevent Forgetting**
**Regularization Methods**
Penalize changes to important weights:
```python
# Elastic Weight Consolidation (EWC)
def ewc_loss(model, importance, old_params, lambda_):
loss = 0
for name, param in model.named_parameters():
loss += (importance[name] * (param - old_params[name])**2).sum()
return lambda_ * loss
# Add to training loss
total_loss = task_loss + ewc_loss(model, fisher, prev_params, 1000)
```
**Replay Methods**
Store and replay old examples:
```python
class ReplayBuffer:
def __init__(self, size_per_task=100):
self.buffer = []
self.size_per_task = size_per_task
def add_task(self, task_data):
samples = random.sample(task_data, self.size_per_task)
self.buffer.extend(samples)
def get_replay_batch(self, size):
return random.sample(self.buffer, size)
```
**Architecture Methods**
Add new capacity for new tasks:
```python
# Progressive networks: Add new column per task
# PackNet: Prune and freeze for each task
# Modular networks: Route to task-specific experts
```
**Comparison**
| Method | Memory | Compute | Performance |
|--------|--------|---------|-------------|
| EWC | Low | Medium | Medium |
| Replay | Medium | Low | High |
| Progressive | High | Low | High |
| PackNet | Low | Low | Medium |
**Metrics**
| Metric | Definition |
|--------|------------|
| Accuracy | Performance on current task |
| Backward transfer | Effect on old tasks |
| Forward transfer | Effect on learning new tasks |
| Forgetting | Accuracy drop on old tasks |
**Use Cases**
- Chatbots learning from conversations
- Robots adapting to new environments
- Recommendation systems evolving with trends
- Any scenario with sequential data streams
**Best Practices**
- Evaluate on all tasks, not just current
- Use replay buffers when storage allows
- Consider task similarity for transfer
- Monitor for catastrophic forgetting