episode-based training

**Episode-based training (episodic training)** is the **standard training paradigm** for meta-learning and few-shot learning, where models learn from sequences of **simulated few-shot tasks called episodes** rather than from individual labeled examples. **The Core Idea** - **Train Like You Test**: Training episodes are structured identically to test-time evaluation — the model practices solving few-shot tasks thousands of times during training. - **Learn to Learn**: Instead of memorizing specific classes, the model learns a **general strategy** for classifying new categories from few examples. - **Task Distribution**: The model samples from a **distribution of tasks** rather than a fixed dataset, learning transferable skills. **Episode Construction** - **Step 1 — Sample Classes**: Randomly select **N classes** from the training class pool (creating an N-way task). These classes change every episode. - **Step 2 — Create Support Set**: For each selected class, sample **K examples** as the support set (K-shot). These are the "training" examples for this episode. - **Step 3 — Create Query Set**: Sample additional examples from the same N classes as the query set. These are the "test" examples. - **Step 4 — Predict & Update**: The model uses the support set to classify query examples. Loss on query predictions drives gradient updates. **Example: 5-Way 5-Shot Episode** - Random 5 classes selected (e.g., dog, cat, bird, fish, car). - **Support set**: 5 images per class = 25 total labeled examples. - **Query set**: 15 images per class = 75 total test examples. - Model sees support images, classifies query images, and loss is computed. - Next episode: 5 completely different classes are selected. **Why Episodic Training Works** - **Alignment**: Training objective matches test-time task structure — no train-test mismatch. - **Diversity**: Each episode presents a different classification problem — prevents memorization of specific classes. - **Generalization Pressure**: The model must develop strategies that work across many different class combinations. **Training Mechanics** - **Outer Loop**: Sample episodes and update model parameters based on episode performance. - **Inner Loop** (for MAML): Adapt model to each episode's support set using gradient descent, then evaluate on queries. - **Batch of Episodes**: Process multiple episodes per gradient step for stable training. **Variations** - **Curriculum Learning**: Start with easier episodes (common classes, more examples) and gradually increase difficulty. - **Task Augmentation**: Apply data augmentations differently across episodes to increase task diversity. - **Mixed Episodic-Batch Training**: Combine episode-based meta-learning with standard batch classification to stabilize training and improve base feature quality. - **Incremental Episodes**: Progressively add classes within an episode to simulate class-incremental learning. **Limitations** - **Sampling Variance**: Random episode sampling can lead to high training variance — some episodes are much harder than others. - **Computational Cost**: Constructing and processing thousands of episodes adds overhead compared to standard batch training. - **Class Imbalance**: Random sampling may over-represent common classes and under-represent rare ones. Episodic training is the **cornerstone of meta-learning** — by practicing few-shot tasks thousands of times during training, models develop robust strategies for rapid learning that transfer to entirely new classes at test time.