Few-Shot Learning - Prototypical Networks

# Few-Shot Learning - Prototypical Networks

## Introduction & Motivation

Few-shot learning: learn from minimal examples. Prototypical networks, metric learning. Applications: rapid adaptation, low-data scenarios.

Motivation: Enable learning from just a few samples.

Applications: Personalization, rapid adaptation, meta-learning.

---

## Core Concepts & Theory

### Prototypes

Class representative in embedding space.

### Metric Learning

Learn distance metric.

### Episode Training

Mini-batch learning simulation.

### Support and Query

Train and test split in episode.

---

## Mathematical Formulation

Prototype:
$$c_k = \frac{1}{|S_k|} \sum_{(x,y) \in S_k} f(x)$$

Distance:
$$d(x, c_k) = -||f(x) - c_k||^2$$

Loss:
$$\mathcal{L} = -\sum_x \log \frac{\exp(d(x, c_y))}{\sum_k \exp(d(x, c_k))}$$

---

## Advanced Theory & Extensions

### Matching Networks

Attention-based matching.

### Siamese Networks

Twin networks for similarity.

### Relation Networks

Learn similarity function.

---

## Computational Considerations

Embedding: O(D²).

Distance: O(k·D).

Episode: O(N·D²).

---

## Practical Implementation Strategies

### Episode Sampling

Construct mini-tasks.

### Metric Learning

Optimize embedding space.

### Augmentation

Regularize with data augmentation.

---

## Benchmark Datasets & Evaluation

omniglot: Character recognition.

mini-ImageNet: Few-shot classification.

CUB: Fine-grained birds.

---

## Key Challenges & Limitations

### Domain Shift

Transfer to new domains.

### Class Imbalance

Uneven class samples.

### Scalability

Slow with many classes.

---

## Hyperparameter Tuning

Embedding dimension: 64-256.

Way (classes): 5-20.

Shot (examples): 1-5.

---

## Real-World Applications & Case Studies

Personalization: User-specific adaptation.

Open-Set Recognition: Recognize new classes.

Transfer Learning: Rapid domain adaptation.

---

## Integration with Other Methods

Few-shot + meta-learning; + transfer learning.

---

## Summary & Key Takeaways

Prototypical networks enable few-shot learning.

Principles:
1. Prototypes: Class representatives.
2. Metric: Learned distance.
3. Episodes: Task simulation.
4. Efficiency: Few examples needed.
5. Scalability: Works across domains.

---

## Appendix: Practical Labs

### Lab 1: Prototype Computation

import numpy as np

def compute_prototypes(support_set, support_labels, embedding_fn, num_classes):
 """Compute class prototypes"""
 prototypes = []
 for c in range(num_classes):
 mask = support_labels == c
 class_emb = embedding_fn(support_set[mask])
 prototype = np.mean(class_emb, axis=0)
 prototypes.append(prototype)
 return np.array(prototypes)

np.random.seed(42)
support = np.random.randn(20, 10)
labels = np.repeat([0, 1, 2, 3, 4], 4)
emb = lambda x: x @ np.random.randn(10, 64)
protos = compute_prototypes(support, labels, emb, 5)
assert protos.shape == (5, 64)
print("✓ Prototype computation working")

### Lab 2: Distance Computation

import numpy as np

def prototypical_distances(query_embedding, prototypes, metric='euclidean'):
 """Compute distances to prototypes"""
 if metric == 'euclidean':
 distances = np.sum((query_embedding - prototypes)**2, axis=1, keepdims=True)
 distances = -np.sqrt(distances)
 elif metric == 'cosine':
 query_norm = query_embedding / (np.linalg.norm(query_embedding) + 1e-8)
 proto_norm = prototypes / (np.linalg.norm(prototypes, axis=1, keepdims=True) + 1e-8)
 distances = query_norm @ proto_norm.T
 return distances

np.random.seed(42)
query = np.random.randn(64)
protos = np.random.randn(5, 64)
dists = prototypical_distances(query, protos)
assert dists.shape == (1, 5)
print("✓ Distance computation working")

### Lab 3: Episode Simulation

import numpy as np

def create_episode(data, labels, num_way=5, num_shot=5, num_query=15):
 """Create few-shot episode"""
 unique_labels = np.unique(labels)
 selected_classes = np.random.choice(unique_labels, num_way, replace=False)
 
 support = []
 query = []
 for cls in selected_classes:
 mask = labels == cls
 indices = np.where(mask)[0]
 
 support_idx = np.random.choice(indices, num_shot, replace=False)
 remaining = np.setdiff1d(indices, support_idx)
 query_idx = np.random.choice(remaining, min(num_query, len(remaining)), replace=False)
 
 support.extend(data[support_idx])
 query.extend(data[query_idx])
 
 return np.array(support), np.array(query)

np.random.seed(42)
data = np.random.randn(100, 10)
labels = np.repeat(range(10), 10)
supp, qu = create_episode(data, labels)
assert supp.shape[0] == 25 # 5-way, 5-shot
print("✓ Episode creation working")

### Lab 4: Classification Accuracy

import numpy as np

def few_shot_accuracy(query_embed, query_labels, prototypes):
 """Compute few-shot accuracy"""
 distances = query_embed @ prototypes.T
 predictions = np.argmax(distances, axis=1)
 accuracy = np.mean(predictions == query_labels)
 return accuracy

np.random.seed(42)
query = np.random.randn(20, 64)
labels = np.random.randint(0, 5, 20)
protos = np.random.randn(5, 64)
acc = few_shot_accuracy(query, labels, protos)
assert 0 <= acc <= 1
print(f"✓ Accuracy: {acc:.2%}")

---