catastrophic interference

**Catastrophic interference** (also called **catastrophic forgetting**) is the phenomenon where a neural network trained on a new task **abruptly and severely forgets** previously learned knowledge. It is the central challenge of **continual learning** — standard neural networks are fundamentally poor at accumulating knowledge across sequential tasks. **Why It Happens** - **Shared Weights**: Neural networks store all knowledge in the same set of weights. When weights are updated for a new task, the changes **overwrite** information stored for previous tasks. - **Gradient Descent**: Optimization moves weights in whatever direction minimizes loss on the current task, with no constraint to preserve performance on old tasks. - **No Explicit Memory**: Unlike human brains, standard neural networks have no mechanism to consolidate and protect important memories. **Examples** - A model trained on **Task A** (classifying animals) then trained on **Task B** (classifying vehicles) may lose the ability to classify animals entirely. - Fine-tuning a pre-trained LLM for one specific task can degrade its general capabilities. - An AI agent learning new skills may suddenly lose previously mastered skills. **Mitigation Strategies** - **Regularization-Based**: **EWC (Elastic Weight Consolidation)** identifies weights important for previous tasks and penalizes changes to them. Other methods: SI (Synaptic Intelligence), MAS (Memory Aware Synapses). - **Replay-Based**: **Experience replay** stores examples from old tasks and replays them during new task training to maintain old knowledge. - **Architecture-Based**: **Progressive neural networks** add new capacity for each task rather than reusing existing weights. **PackNet** uses weight pruning to allocate subnetworks per task. - **Knowledge Distillation**: Use the model's own outputs on old tasks as soft targets (teacher) while learning new tasks. **Relevance to LLMs** - Fine-tuning LLMs can cause catastrophic forgetting of general knowledge — mitigated by **LoRA** (which modifies only a small subset of parameters) and **careful learning rate selection**. - **RLHF** can cause forgetting of pre-training knowledge — known as the **alignment tax**. Catastrophic interference is the **fundamental barrier** to building AI systems that learn continuously — overcoming it is essential for lifelong learning systems.