Meta-Learning (MAML and Variants) is the "learning to learn" paradigm that trains a model across a distribution of tasks so that it acquires an initialization (or learning strategy) capable of adapting to entirely new tasks from only a handful of labeled examples — achieving few-shot generalization without task-specific retraining from scratch.

The Few-Shot Problem

Conventional deep learning requires thousands to millions of labeled examples per class. In robotics, medical imaging, drug discovery, and rare-event detection, collecting more than 1-5 examples per class is often impossible. Meta-learning reframes the objective: instead of learning a single task well, learn a prior over tasks that enables rapid adaptation.

How MAML Works

Model-Agnostic Meta-Learning uses a bi-level optimization:

• Inner Loop (Task Adaptation): For each sampled task (e.g., classify 5 new animal species from 5 examples each), take 1-5 gradient steps from the current initialization on the task's support set (the few labeled examples). This produces a task-specific adapted model.
• Outer Loop (Meta-Update): Evaluate the adapted model on the task's query set (held-out examples). Backpropagate through the inner loop steps to update the shared initialization so that future inner-loop adaptations produce better query-set performance.

After meta-training across hundreds of tasks, the initialization sits at a point in parameter space from which a small number of gradient steps can reach a good solution for any task from the training distribution.

Variants and Extensions

• Reptile: A first-order approximation that avoids computing second-order gradients through the inner loop. Simpler to implement, nearly matching MAML accuracy.
• ProtoNet (Prototypical Networks): A metric-learning approach that embeds support examples into a space and classifies query examples by distance to class centroids. No inner-loop gradient computation — fast and stable.
• ANIL (Almost No Inner Loop): Shows that most of MAML's benefit comes from the learned feature extractor, not inner-loop adaptation of all layers. Only the final classification head is adapted in the inner loop.

Practical Considerations

MAML's second-order gradients are memory-intensive and can destabilize training for large models. First-order approximations (Reptile, FO-MAML) trade a small accuracy reduction for 2-3x memory savings. Task construction quality — ensuring meta-training tasks mirror the distribution of expected deployment tasks — has more impact on final few-shot accuracy than the choice of meta-learning algorithm.

Meta-Learning is the principled solution to the data scarcity problem — encoding the structure of how to learn efficiently into the model's initialization so that a handful of examples is all it takes to master a new concept.