Few-Shot Learning Prototypical Networks Meta-Learning
# Few-Shot Learning: Prototypical Networks & Meta-Learning
## Introduction & Motivation
Few-Shot Learning: learn from few examples per class. Prototypical Networks: class prototypes as centroids. Meta-learning: learn to learn. Applications: rapid adaptation, low-data domains.
Motivation: Learn new tasks with minimal examples.
Applications: New class adaptation, few-example scenarios.
---
## Core Concepts & Theory
### Support Set
Few labeled examples per class.
### Query Set
Test examples on learned task.
### Class Prototypes
Mean embedding per class.
---
## Mathematical Formulation
Prototypical networks:
$$c_k = \frac{1}{|S_k|} \sum_{(x_i, y_i) \in S_k} f_ heta(x_i)$$
Distance to prototype:
$$d(x, c_k) = \|f_ heta(x) - c_k\|^2$$
Prediction:
$$P(y=k | x) = \frac{\exp(-d(x, c_k))}{\sum_j \exp(-d(x, c_j))}$$
---
## Advanced Theory & Extensions
### Matching Networks
Attention-based matching.
### Relation Networks
Learned metric function.
### Model-Agnostic Meta-Learning (MAML)
Gradient-based meta-learning.
---
## Computational Considerations
Support set: O(K·N·D) K classes, N shots.
Query: O(Q·K·D) Q queries, K classes.
Metric learning: O((K·N + Q)²) pairwise.
---
## Practical Implementation Strategies
### Episodic Training
Sample tasks during training.
### Data Augmentation
Increase effective samples.
### Metric Learning
Initialize with pretrained features.
---
## Benchmark Datasets & Evaluation
miniImageNet: 5-way 5-shot benchmark.
Omniglot: Small dataset; quick adaptation.
tieredImageNet: Large-scale few-shot.
---
## Key Challenges & Limitations
### Intra-class Variance
High variance in few examples.
### Class Imbalance
Unequal per-class samples.
### Distribution Shift
Support and query mismatch.
---
## Hyperparameter Tuning
Number of shots (N): 1-10; data availability.
Number of ways (K): 5-20; task difficulty.
Learning rate: 1e-3 to 1e-4.
---
## Real-World Applications & Case Studies
Image Recognition: Few new object classes.
Medical Imaging: Rare disease diagnosis.
Personalization: User-specific adaptation.
---
## Integration with Other Methods
Few-shot + Metric Learning → better distances.
Few-shot + Transfer → task adaptation.
---
## Summary & Key Takeaways
Few-Shot Learning via prototypical networks enables rapid task adaptation through class prototypes and metric learning from minimal examples.
Principles:
1. Support set: few labeled examples.
2. Prototypes: class centroids.
3. Metric: distance-based classification.
4. Episodic: task sampling.
5. Meta-learning: learn to adapt.
---
---
## Appendix: Practical Labs
### Lab 1: Prototypical Network
import numpy as np
def prototypical_network(support_features, support_labels, query_features, num_classes):
"""Prototypical Networks inference"""
# Compute class prototypes
prototypes = np.zeros((num_classes, support_features.shape[1]))
for c in range(num_classes):
class_features = support_features[support_labels == c]
prototypes[c] = class_features.mean(axis=0)
# Compute distances
distances = np.zeros((len(query_features), num_classes))
for i, query in enumerate(query_features):
for c in range(num_classes):
distances[i, c] = np.linalg.norm(query - prototypes[c])
# Classify as nearest prototype
predictions = np.argmin(distances, axis=1)
return predictions, prototypes
# Test
np.random.seed(42)
support_features = np.random.randn(20, 64)
support_labels = np.repeat(np.arange(5), 4) # 5 classes, 4 shots
query_features = np.random.randn(10, 64)
preds, protos = prototypical_network(support_features, support_labels, query_features, 5)
assert preds.shape == (10,), "Predictions shape"
assert protos.shape == (5, 64), "Prototypes shape"
print("✓ Prototypical networks working")
if __name__ == "__main__":
print("Lab 1: PrototypicalNetworks - PASSED")### Lab 2: Episode Sampling
import numpy as np
def sample_episode(data, labels, num_classes=5, num_shots=5, num_queries=15):
"""Sample few-shot learning episode"""
# Select classes
available_classes = np.unique(labels)
selected_classes = np.random.choice(available_classes, num_classes, replace=False)
support_features = []
support_labels_ep = []
query_features = []
query_labels_ep = []
for new_label, class_id in enumerate(selected_classes):
class_indices = np.where(labels == class_id)[0]
selected_indices = np.random.choice(class_indices, num_shots + num_queries, replace=False)
support_features.append(data[selected_indices[:num_shots]])
support_labels_ep.extend([new_label] * num_shots)
query_features.append(data[selected_indices[num_shots:]])
query_labels_ep.extend([new_label] * num_queries)
support_features = np.vstack(support_features)
query_features = np.vstack(query_features)
return support_features, np.array(support_labels_ep), query_features, np.array(query_labels_ep)
# Test
np.random.seed(42)
data = np.random.randn(100, 64)
labels = np.repeat(np.arange(10), 10)
supp_f, supp_l, query_f, query_l = sample_episode(data, labels)
assert supp_f.shape[0] == 25, "Support size"
assert query_f.shape[0] == 75, "Query size"
print("✓ Episode sampling working")
if __name__ == "__main__":
print("Lab 2: EpisodeSampling - PASSED")### Lab 3: Few-Shot Accuracy
import numpy as np
def few_shot_accuracy(predictions, labels):
"""Compute few-shot accuracy"""
correct = (predictions == labels).sum()
total = len(labels)
accuracy = correct / total
return accuracy
# Test
predictions = np.array([0, 1, 2, 0, 1, 2, 0, 1, 2])
labels = np.array([0, 1, 2, 0, 1, 1, 0, 1, 2])
acc = few_shot_accuracy(predictions, labels)
assert 0 <= acc <= 1, "Accuracy in [0,1]"
print("✓ Few-shot accuracy working")
if __name__ == "__main__":
print("Lab 3: FewShotAccuracy - PASSED")### Lab 4: Metric Distance Functions
import numpy as np
def compute_distances(query, support, metric='euclidean'):
"""Compute distances between query and support samples"""
if metric == 'euclidean':
distances = np.linalg.norm(query[None, :] - support, axis=1)
elif metric == 'cosine':
q_norm = query / (np.linalg.norm(query) + 1e-8)
s_norm = support / (np.linalg.norm(support, axis=1, keepdims=True) + 1e-8)
distances = 1 - (q_norm @ s_norm.T)
else:
raise ValueError(f"Unknown metric: {metric}")
return distances
# Test
query = np.random.randn(64)
support = np.random.randn(20, 64)
dist_eucl = compute_distances(query, support, 'euclidean')
dist_cosine = compute_distances(query, support, 'cosine')
assert dist_eucl.shape == (20,), "Distance shape"
assert dist_cosine.shape == (20,), "Cosine shape"
print("✓ Distance functions working")
if __name__ == "__main__":
print("Lab 4: DistanceFunctions - PASSED")