Meta-Learning and Few-Shot Adaptation

# Meta-Learning and Few-Shot Adaptation

## Introduction & Motivation

Meta-learning enables rapid adaptation to new tasks with minimal data. Critical for engineering applications with limited labeled examples where transferring learned "learning strategies" accelerates discovery across related problems.

Motivation: Learn to learn efficiently from few examples.

Applications: Few-shot learning, domain adaptation, rapid model development, transfer learning.

---

## Core Concepts & Theory

### Task Distribution

Related tasks sampling.

### Inner Loop

Task-specific adaptation.

### Outer Loop

Meta-parameter optimization.

### Support/Query Sets

Few-shot evaluation protocol.

---

## Mathematical Formulation

MAML:
$$ heta^* = heta - \alpha abla_ heta L_{task}( heta)$$

Meta-Update:
$$ heta \leftarrow heta - \beta abla_ heta L_{meta}( heta^*)$$

Few-Shot Loss:
$$L = \frac{1}{T} \sum_{ au} L_{ au}(D_{support}, D_{query})$$

---

## Advanced Theory & Extensions

### Prototypical Networks

Metric learning approach.

### Relation Networks

Learning comparison metrics.

### Conditional Networks

Task-conditional parameters.

---

## Computational Considerations

Inner Loop: O(α·D²) adaptation.

Outer Loop: O(T·α·D²) for T tasks.

Total: O(T·α·D²) per meta-batch.

---

## Practical Implementation Strategies

### Task Sampling

Stratified task selection.

### Inner Loop Steps

Adaptation depth.

### Meta-Batch Size

Task count per update.

---

## Benchmark Datasets & Evaluation

Omniglot: Few-shot recognition.

miniImageNet: Image classification.

Domain Tasks: Engineering applications.

---

## Key Challenges & Limitations

### Task Similarity

Performance on dissimilar tasks.

### Inner Loop Convergence

Limited adaptation steps.

### Computational Cost

Multiple nested optimizations.

---

## Hyperparameter Tuning

Inner LR (α): 0.01-0.1.

Outer LR (β): 1e-4 to 1e-3.

Adaptation steps: 1-5.

---

## Real-World Applications & Case Studies

Materials Property: Few examples.

Sensor Calibration: Rapid adaptation.

Process Control: New equipment.

---

## Integration with Other Methods

Meta-learning + transfer learning; + neural networks; + optimization.

---

## Summary & Key Takeaways

Meta-learning accelerates task-specific learning.

Principles:
1. Task Distribution: Sample related tasks.
2. Inner Loop: Task adaptation.
3. Outer Loop: Meta-optimization.
4. Support Set: Few examples.
5. Query: Evaluation protocol.

---

## Appendix: Practical Labs

### Lab 1: MAML Training

import numpy as np

class MAMLLearner:
 def __init__(self, n_features=10):
 self.meta_theta = np.random.randn(n_features) * 0.1
 
 def adapt(self, X_support, y_support, alpha=0.01, steps=1):
 """Inner loop adaptation"""
 theta = self.meta_theta.copy()
 
 for _ in range(steps):
 pred = X_support @ theta
 grad = X_support.T @ (pred - y_support)
 theta -= alpha * grad
 
 return theta
 
 def meta_update(self, tasks, beta=0.001):
 """Outer loop meta-update"""
 meta_grad = np.zeros_like(self.meta_theta)
 
 for X_sup, y_sup, X_qry, y_qry in tasks:
 adapted_theta = self.adapt(X_sup, y_sup)
 
 pred_qry = X_qry @ adapted_theta
 task_grad = X_qry.T @ (pred_qry - y_qry)
 meta_grad += task_grad
 
 self.meta_theta -= beta * meta_grad / len(tasks)

print(f"✓ MAML configured")

### Lab 2: Prototypical Networks

import numpy as np

class PrototypicalNetworks:
 def __init__(self):
 self.prototypes = {}
 
 def compute_prototypes(self, X_support, y_support):
 """Compute class prototypes"""
 classes = np.unique(y_support)
 
 for c in classes:
 mask = y_support == c
 self.prototypes[c] = np.mean(X_support[mask], axis=0)
 
 def predict(self, X_query):
 """Classify by nearest prototype"""
 predictions = np.zeros(len(X_query))
 
 for i, x in enumerate(X_query):
 distances = {c: np.linalg.norm(x - p) for c, p in self.prototypes.items()}
 predictions[i] = min(distances, key=distances.get)
 
 return predictions

print(f"✓ Prototypical networks implemented")

### Lab 3: Few-Shot Learning

import numpy as np

class FewShotLearner:
 def __init__(self, n_way=3, k_shot=2):
 self.n_way = n_way
 self.k_shot = k_shot
 self.model = np.random.randn(10, 1) * 0.1
 
 def create_few_shot_task(self, X, y):
 """Create support and query sets"""
 support_X, support_y = [], []
 query_X, query_y = [], []
 
 for c in range(self.n_way):
 class_indices = np.where(y == c)[0]
 
 support_idx = class_indices[:self.k_shot]
 query_idx = class_indices[self.k_shot:self.k_shot+1]
 
 support_X.extend(X[support_idx])
 support_y.extend([c] * len(support_idx))
 query_X.extend(X[query_idx])
 query_y.extend([c] * len(query_idx))
 
 return np.array(support_X), np.array(support_y), np.array(query_X), np.array(query_y)

print(f"✓ Few-shot task creation configured")

### Lab 4: Rapid Adaptation

import numpy as np

class RapidAdaptor:
 def __init__(self, base_model_dim=10):
 self.base_model = np.random.randn(base_model_dim, 1) * 0.1
 
 def quick_adapt(self, X_few, y_few, adaptation_steps=3):
 """Quick adaptation to new task"""
 model = self.base_model.copy()
 
 for _ in range(adaptation_steps):
 pred = X_few @ model
 error = pred - y_few.reshape(-1, 1)
 grad = X_few.T @ error
 
 model -= 0.1 * grad / (np.linalg.norm(grad) + 1e-8)
 
 return model
 
 def evaluate_adaptation(self, X_test, y_test, adapted_model):
 """Evaluate adapted model"""
 pred = X_test @ adapted_model
 mse = np.mean((pred - y_test.reshape(-1, 1)) ** 2)
 return mse

adaptor = RapidAdaptor()

X_few = np.random.randn(3, 10)
y_few = np.random.rand(3)
adapted = adaptor.quick_adapt(X_few, y_few)

X_test = np.random.randn(10, 10)
y_test = np.random.rand(10)
mse = adaptor.evaluate_adaptation(X_test, y_test, adapted)

print(f"✓ Adaptation MSE: {mse:.4f}")

---