meta learning maml

**Meta-Learning (MAML and Variants)** is the **"learning to learn" paradigm that trains a model across a distribution of tasks so that it acquires an initialization (or learning strategy) capable of adapting to entirely new tasks from only a handful of labeled examples — achieving few-shot generalization without task-specific retraining from scratch**. **The Few-Shot Problem** Conventional deep learning requires thousands to millions of labeled examples per class. In robotics, medical imaging, drug discovery, and rare-event detection, collecting more than 1-5 examples per class is often impossible. Meta-learning reframes the objective: instead of learning a single task well, learn a prior over tasks that enables rapid adaptation. **How MAML Works** Model-Agnostic Meta-Learning uses a bi-level optimization: - **Inner Loop (Task Adaptation)**: For each sampled task (e.g., classify 5 new animal species from 5 examples each), take 1-5 gradient steps from the current initialization on the task's support set (the few labeled examples). This produces a task-specific adapted model. - **Outer Loop (Meta-Update)**: Evaluate the adapted model on the task's query set (held-out examples). Backpropagate through the inner loop steps to update the shared initialization so that future inner-loop adaptations produce better query-set performance. After meta-training across hundreds of tasks, the initialization sits at a point in parameter space from which a small number of gradient steps can reach a good solution for any task from the training distribution. **Variants and Extensions** - **Reptile**: A first-order approximation that avoids computing second-order gradients through the inner loop. Simpler to implement, nearly matching MAML accuracy. - **ProtoNet (Prototypical Networks)**: A metric-learning approach that embeds support examples into a space and classifies query examples by distance to class centroids. No inner-loop gradient computation — fast and stable. - **ANIL (Almost No Inner Loop)**: Shows that most of MAML's benefit comes from the learned feature extractor, not inner-loop adaptation of all layers. Only the final classification head is adapted in the inner loop. **Practical Considerations** MAML's second-order gradients are memory-intensive and can destabilize training for large models. First-order approximations (Reptile, FO-MAML) trade a small accuracy reduction for 2-3x memory savings. Task construction quality — ensuring meta-training tasks mirror the distribution of expected deployment tasks — has more impact on final few-shot accuracy than the choice of meta-learning algorithm. Meta-Learning is **the principled solution to the data scarcity problem** — encoding the structure of how to learn efficiently into the model's initialization so that a handful of examples is all it takes to master a new concept.