continual learning

**Continual Learning and Catastrophic Forgetting — Training Neural Networks Across Sequential Tasks** Continual learning addresses the fundamental challenge of training neural networks on a sequence of tasks without forgetting previously acquired knowledge. Catastrophic forgetting, where learning new information overwrites old representations, remains one of the most significant obstacles to building truly adaptive AI systems that learn throughout their operational lifetime. — **The Catastrophic Forgetting Problem** — Understanding why neural networks forget is essential to developing effective continual learning strategies: - **Parameter overwriting** occurs when gradient updates for new tasks modify weights critical to previous task performance - **Representation drift** shifts internal feature representations away from configurations optimal for earlier tasks - **Distribution shift** between sequential tasks forces the network to adapt to changing input-output relationships - **Capacity limitations** mean finite-parameter networks must balance representational resources across all learned tasks - **Stability-plasticity dilemma** captures the fundamental tension between retaining old knowledge and acquiring new capabilities — **Regularization-Based Approaches** — These methods constrain weight updates to protect parameters important for previously learned tasks: - **Elastic Weight Consolidation (EWC)** uses Fisher information to identify and penalize changes to task-critical parameters - **Synaptic Intelligence (SI)** tracks parameter importance online during training based on contribution to loss reduction - **Memory Aware Synapses (MAS)** estimates importance through sensitivity of the learned function to parameter perturbations - **Progressive neural networks** freeze previous task columns and add lateral connections for new tasks - **PackNet** iteratively prunes and freezes subnetworks for each task, allocating remaining capacity to future tasks — **Replay and Rehearsal Methods** — Replay-based strategies maintain access to previous task data through storage or generation: - **Experience replay** stores a small buffer of examples from previous tasks and interleaves them during new task training - **Generative replay** trains a generative model to produce synthetic examples from previous task distributions - **Gradient episodic memory (GEM)** constrains gradient updates to avoid increasing loss on stored exemplars - **Dark experience replay** stores and replays model logits alongside input examples for knowledge distillation - **Coreset selection** identifies maximally informative subsets of previous data for efficient memory buffer utilization — **Architecture-Based Solutions** — Structural approaches modify the network architecture to accommodate new tasks while preserving existing capabilities: - **Dynamic expandable networks** grow the architecture by adding neurons or layers when existing capacity is insufficient - **Task-specific modules** route inputs through dedicated subnetworks based on task identity or learned routing - **Hypernetwork approaches** use a meta-network to generate task-specific weight configurations on demand - **Modular networks** compose shared and task-specific components to balance knowledge sharing and interference avoidance - **Sparse coding** activates different sparse subsets of neurons for different tasks to minimize representational overlap — **Evaluation Protocols and Metrics** — Rigorous assessment of continual learning requires standardized benchmarks and comprehensive metrics: - **Average accuracy** measures mean performance across all tasks after the complete learning sequence - **Backward transfer** quantifies how much learning new tasks improves or degrades performance on previous tasks - **Forward transfer** assesses whether knowledge from previous tasks accelerates learning on subsequent tasks - **Forgetting measure** tracks the maximum performance drop on each task relative to its peak accuracy - **Task-incremental vs class-incremental** settings differ in whether task identity is provided at inference time **Continual learning remains a frontier challenge in deep learning, with practical implications for deployed systems that must adapt to evolving data distributions, and solving catastrophic forgetting is widely considered essential for achieving artificial general intelligence.**