continual learning
**Continual Learning** is the **ability of neural networks to learn new tasks sequentially without forgetting previously learned knowledge** — addressing the catastrophic forgetting problem that causes neural networks to lose old information when trained on new tasks.
**Catastrophic Forgetting**
- Standard neural networks: When fine-tuned on new task → overwrites weights that encoded old task.
- Example: Fine-tune ImageNet model on medical images → ImageNet accuracy drops 40%.
- Biological memory: Doesn't forget old skills when learning new ones (complementary learning systems).
**Continual Learning Strategies**
**Regularization-Based**:
- **EWC (Elastic Weight Consolidation)**: Add penalty that protects important weights.
- $L = L_{new} + \lambda \sum_i F_i(\theta_i - \theta_i^*)^2$
- $F_i$: Fisher information — importance of parameter $i$ for old task.
- Important weights for old task → penalized from moving far.
- **SI (Synaptic Intelligence)**: Online importance estimation during training.
- Limitation: Memory scales O(tasks × params) for task importance storage.
**Memory Replay**:
- Store examples from old tasks → replay during new task training.
- **Experience Replay**: Real stored samples. Memory cost: grows with tasks.
- **Generative Replay (DGR)**: Train generative model on old data → replay synthetic samples.
- **GDumb**: Simply train on memory buffer — surprisingly competitive baseline.
**Architecture-Based**:
- **Progressive Neural Networks**: New column per task, lateral connections from old columns.
- Zero forgetting, but grows in size.
- **PackNet**: Prune old task → use freed capacity for new task.
**Prompt-Based Continual Learning**:
- Freeze pretrained model; learn small prompts per task.
- L2P (Learning to Prompt): Shared prompt pool — tasks select relevant prompts.
- No forgetting of pretrained features; task-specific adaptation via prompts.
Continual learning is **a fundamental requirement for AI systems deployed in changing environments** — industrial robots learning new assembly tasks, medical models adapting to new diseases, and personal assistants adapting to individual users all require learning new things without erasing old knowledge.