gradient episodic memory
**Gradient episodic memory** is **a continual-learning algorithm that constrains new-task gradients so they do not increase loss on stored past-task examples** - Projected gradients enforce non-interference conditions using episodic memory constraints.
**What Is Gradient episodic memory?**
- **Definition**: A continual-learning algorithm that constrains new-task gradients so they do not increase loss on stored past-task examples.
- **Core Mechanism**: Projected gradients enforce non-interference conditions using episodic memory constraints.
- **Operational Scope**: It is applied during data scheduling, parameter updates, or architecture design to preserve capability stability across many objectives.
- **Failure Modes**: Constraint solving can increase training cost and become complex at larger task counts.
**Why Gradient episodic memory Matters**
- **Retention and Stability**: It helps maintain previously learned behavior while new tasks are introduced.
- **Transfer Efficiency**: Strong design can amplify positive transfer and reduce duplicate learning across tasks.
- **Compute Use**: Better task orchestration improves return from fixed training budgets.
- **Risk Control**: Explicit monitoring reduces silent regressions in legacy capabilities.
- **Program Governance**: Structured methods provide auditable rules for updates and rollout decisions.
**How It Is Used in Practice**
- **Design Choice**: Select the method based on task relatedness, retention requirements, and latency constraints.
- **Calibration**: Set memory budgets and projection tolerances with ablations that measure retention versus compute overhead.
- **Validation**: Track per-task gains, retention deltas, and interference metrics at every major checkpoint.
Gradient episodic memory is **a core method in continual and multi-task model optimization** - It provides explicit optimization safeguards against catastrophic forgetting.