rehearsal methods

**Rehearsal methods** (also called **replay methods**) are continual learning techniques that combat catastrophic forgetting by **storing and periodically replaying examples** from previously learned tasks while training on new tasks. They are among the most effective approaches to continual learning. **Core Idea** - Maintain a **memory buffer** containing representative examples from past tasks. - When training on a new task, interleave new task data with replayed examples from the buffer. - This ensures the model continues to see old data, preventing weights from drifting away from solutions that work for previous tasks. **Types of Rehearsal** - **Exact Replay**: Store actual training examples from previous tasks. Simple and effective but requires memory for storing raw data. - **Generative Replay**: Train a generative model (GAN, VAE) on previous task data and use it to **generate synthetic examples** for replay. No need to store real data, but the quality of generated examples matters. - **Feature Replay**: Store intermediate feature representations rather than raw inputs. More compact than raw data storage. - **Gradient-Based Replay**: Store gradient information from previous tasks and use it to constrain learning on new tasks (e.g., **GEM — Gradient Episodic Memory**). **Key Design Decisions** - **Buffer Size**: How many examples to store. Larger buffers preserve more information but consume more memory. - **Example Selection**: Which examples to keep in the buffer (see exemplar selection strategies). - **Replay Ratio**: How often to replay old examples relative to new data. Too little replay → forgetting; too much → slow learning on new tasks. - **Buffer Update**: When to add new examples and which old examples to evict as the buffer fills. **Effectiveness** - Rehearsal methods consistently **outperform regularization-only approaches** (like EWC) on standard continual learning benchmarks. - Even a very small buffer (50–100 examples per class) provides significant forgetting prevention. - Combining rehearsal with regularization further improves results. **Limitations** - **Privacy**: Storing real examples from previous tasks may violate privacy constraints. - **Scalability**: Buffer size grows with the number of tasks (or examples must be evicted). Rehearsal methods are the **most practical and effective** approach to continual learning in production systems — simple exact replay with a well-designed buffer is hard to beat.