Meta-Learning Model-Agnostic Meta-Learning Maml

# Meta-Learning: Model-Agnostic Meta-Learning (MAML)

## Introduction & Motivation

Meta-Learning: learn to learn. MAML: gradient-based adaptation. Few-shot learning via meta-gradient. Applications: rapid adaptation, novel tasks.

Motivation: Learn quickly from few samples; task adaptation.

Applications: Few-shot, rapid learning, adaptation.

---

## Core Concepts & Theory

### Inner Loop

Adapt to task via gradient steps.

### Outer Loop

Optimize for fast adaptation.

### Meta-Gradient

Gradient through gradient computation.

---

## Mathematical Formulation

MAML objective:
$$\min_ heta \sum_{ au} L_{ au}( heta - \alpha abla L_{ au}( heta))$$

Inner loop:
$$ heta'_i = heta - \alpha abla L_i( heta)$$

Outer loop:
$$ heta \leftarrow heta - \beta abla_ heta \sum_i L_i( heta'_i)$$

---

## Summary & Key Takeaways

Meta-Learning via MAML enables rapid task adaptation through gradient-based meta-optimization.

---

---

## Appendix: Practical Labs

### Lab 1: Inner Loop Update

import numpy as np

def inner_loop_update(theta, gradient, alpha=0.01):
 """Single gradient step in inner loop"""
 theta_prime = theta - alpha * gradient
 return theta_prime

# Test
np.random.seed(42)
theta = np.random.randn(100)
grad = np.random.randn(100)

theta_prime = inner_loop_update(theta, grad)

assert theta_prime.shape == theta.shape, "Shape preserved"
print("✓ Inner loop working")

if __name__ == "__main__":
 print("Lab 1: InnerLoop - PASSED")

### Lab 2: Meta-Gradient

import numpy as np

def compute_meta_gradient(task_losses_after_update, theta_step_size=0.01):
 """Compute gradient for outer loop"""
 meta_gradient = np.mean(task_losses_after_update) / theta_step_size
 return meta_gradient

# Test
np.random.seed(42)
losses = np.random.rand(10)

meta_grad = compute_meta_gradient(losses)

assert np.isfinite(meta_grad), "Gradient finite"
print("✓ Meta-gradient working")

if __name__ == "__main__":
 print("Lab 2: MetaGradient - PASSED")

### Lab 3: Few-Shot Task Sampling

import numpy as np

def sample_few_shot_task(data, labels, k_shot=5, k_way=5):
 """Sample few-shot task"""
 classes = np.unique(labels)
 support_x, support_y = [], []
 query_x, query_y = [], []
 
 for c in np.random.choice(classes, k_way, replace=False):
 class_indices = np.where(labels == c)[0]
 selected = np.random.choice(class_indices, k_shot + 15, replace=False)
 
 support_x.append(data[selected[:k_shot]])
 support_y.extend([c] * k_shot)
 query_x.append(data[selected[k_shot:]])
 query_y.extend([c] * 15)
 
 return np.vstack(support_x), np.array(support_y), np.vstack(query_x), np.array(query_y)

# Test
np.random.seed(42)
data = np.random.randn(100, 28, 28)
labels = np.repeat(np.arange(10), 10)

sup_x, sup_y, q_x, q_y = sample_few_shot_task(data, labels)

assert len(sup_y) == 25, "Support size"
print("✓ Few-shot sampling working")

if __name__ == "__main__":
 print("Lab 3: FewShotSampling - PASSED")

### Lab 4: MAML Loss

import numpy as np

def maml_loss(support_loss, query_loss_after_update, meta_alpha=0.01):
 """MAML training objective"""
 total_loss = query_loss_after_update
 return total_loss

# Test
np.random.seed(42)
sup_loss = 0.5
q_loss = 0.6

loss = maml_loss(sup_loss, q_loss)

assert np.isfinite(loss), "Loss finite"
print("✓ MAML loss working")

if __name__ == "__main__":
 print("Lab 4: MAMLLoss - PASSED")