class-incremental learning
**Class-incremental learning (CIL)** is a continual learning scenario where new **output classes** are added over time, and the model must learn to distinguish among **all classes seen so far** — including both old and new ones — without access to data from previous tasks.
**The Challenge**
- **Task 1**: Learn to classify classes {cat, dog}.
- **Task 2**: Now add classes {bird, fish}. The model must classify among {cat, dog, bird, fish} — but only has training data for bird and fish.
- **Task 3**: Add {horse, cow}. The model must handle all 6 classes with only horse and cow data available.
**Why CIL is Hard**
- **Output Space Grows**: The classification head must expand to accommodate new classes, and the model must maintain decision boundaries between all classes.
- **No Task ID at Test Time**: Unlike task-incremental learning, the model doesn't know which task a test example belongs to — it must distinguish among all classes simultaneously.
- **Class Imbalance**: During training on a new task, only new classes have available data, creating severe imbalance that biases the model toward recent classes.
- **Decision Boundary Shift**: As new classes are added, old decision boundaries need adjustment even though old data isn't available.
**Key Methods**
- **iCaRL**: Stores exemplars from old classes and uses **nearest-class-mean** classification in feature space rather than the output layer.
- **LUCIR**: Uses cosine normalization and less-forget constraint to maintain balanced representations.
- **PODNet**: Preserves intermediate representations through **pooled outputs distillation** across spatial dimensions.
- **DER (Dark Experience Replay)**: Stores old examples with their **logits** and uses knowledge distillation during replay.
- **Bias Correction**: Explicitly correct the bias toward new classes in the classification layer.
**Evaluation Protocol**
- Report accuracy on **all seen classes** after each incremental step.
- The key metric is the **average incremental accuracy** — the average of accuracies across all steps.
- Compare against the **joint training** upper bound (training on all data simultaneously).
Class-incremental learning is considered the **hardest** standard continual learning setting and is the most representative of real-world deployment scenarios where new categories continuously emerge.