memory buffer
**A memory buffer** in continual learning is a fixed-size storage that holds **representative examples from previously learned tasks**, enabling rehearsal and preventing catastrophic forgetting. The design of the memory buffer — its size, what it stores, and how it manages capacity — is crucial for continual learning performance.
**What a Memory Buffer Stores**
- **Raw Examples**: The original input-output pairs (x, y). Most straightforward approach.
- **Features**: Intermediate representations from the model — more compact than raw data.
- **Logits**: The model's output distribution (soft labels) at the time the example was stored. Used for knowledge distillation during replay.
- **Gradients**: Gradient vectors from previous tasks, used to constrain optimization direction (e.g., GEM).
**Buffer Management Strategies**
- **Reservoir Sampling**: Each new example has a probability of replacing an existing buffer entry, ensuring the buffer is a uniform random sample of all seen data. Simple and theoretically sound.
- **Class-Balanced**: Maintain equal representation of each class/task in the buffer. Prevents bias toward recent or dominant classes.
- **Herding**: Select examples that best approximate the class mean in feature space — keeps the most representative examples.
- **FIFO (First-In-First-Out)**: Evict the oldest examples. Simple but may lose important early knowledge.
- **Loss-Based**: Keep examples with the highest loss (hardest examples) or most diverse coverage.
**Buffer Size Trade-Offs**
- **Larger Buffer**: Better knowledge retention, higher accuracy on old tasks, but more memory consumption and potential privacy concerns.
- **Smaller Buffer**: Lower memory cost, faster sampling, but more forgetting. Even **20 examples per class** can significantly reduce forgetting.
- **Typical Sizes**: Research benchmarks use 200–5,000 examples total across all tasks.
**Advanced Techniques**
- **Compressed Buffers**: Store compressed representations to fit more examples in the same space.
- **Generative Buffers**: Replace stored examples with a generative model that can produce synthetic examples from old tasks on demand.
- **Dynamic Sizing**: Adjust buffer allocation as the number of tasks grows — each task gets a smaller slice.
The memory buffer is the **heart of rehearsal-based continual learning** — its design directly determines how well the system balances remembering old knowledge with learning new information.