Few-Shot Segmentation
# Few-Shot Segmentation
## Introduction & Motivation
Few-Shot Segmentation: segment new object classes from few examples. Prototype-based, metric learning. Applications: new class adaptation, data-efficient learning.
Motivation: Enable segmentation with minimal training data.
Applications: Rare object detection, fast adaptation to new classes.
---
## Core Concepts & Theory
### Support Set
Few examples of target class.
### Query Set
Images to segment.
### Prototype Learning
Class-specific feature prototypes.
### Metric Learning
Distance-based classification.
---
## Mathematical Formulation
Prototypical Network:
$$p_c = \frac{1}{|S_c|} \sum_{(x_i, y_i) \in S_c} f(x_i, y_i)$$
Distance-Based Segmentation:
$$P(y=c|x) = \frac{\exp(-d(f(x), p_c))}{\sum_k \exp(-d(f(x), p_k))}$$
Support Loss:
$$L = -\sum_c \sum_{x \in Q_c} \log P(y=c|x)$$
---
## Advanced Theory & Extensions
### Meta-Learning
Learn to learn with few examples.
### Task-Aware Adaptation
Customize learning per task.
### Cross-Attention
Match support and query features.
---
## Computational Considerations
Prototype computation: O(|S_c|·feature_dim).
Distance computation: O(|Q|·num_classes·feature_dim).
Meta-learning: O(tasks·episodes).
---
## Practical Implementation Strategies
### Backbone Freezing
Freeze feature extractor.
### Support Augmentation
Augment few examples.
### Episodic Training
Simulate few-shot scenarios.
---
## Benchmark Datasets & Evaluation
PASCAL-5i: Few-shot segmentation split.
COCO-20i: Multi-class segmentation.
Camvid: Autonomous driving splits.
---
## Key Challenges & Limitations
### Domain Shift
Target class differs from base.
### Limited Data
Few pixels available.
### Intra-class Variance
Variation within classes.
---
## Hyperparameter Tuning
Learning rate: 1e-5 to 1e-4.
Support examples: 1-5 per class.
Feature dimension: 256-512.
---
## Real-World Applications & Case Studies
New Object Detection: Adapt to new classes.
Medical Imaging: Segment rare pathologies.
Robotics: Learn new object types.
---
## Integration with Other Methods
Few-shot segmentation + meta-learning for task adaptation; + data augmentation for robustness.
---
## Summary & Key Takeaways
Few-Shot Segmentation via prototypical networks enables adaptation with minimal training data.
Principles:
1. Prototypes: Class-specific features.
2. Metric learning: Distance-based classification.
3. Meta-learning: Task-agnostic optimization.
4. Support set: Few examples per class.
5. Episodic training: Simulate few-shot setting.
---
---
## Appendix: Practical Labs
### Lab 1: Prototype Computation
import numpy as np
def compute_prototypes(support_features, support_labels, num_classes):
"""Compute class prototypes"""
prototypes = []
for c in range(num_classes):
mask = support_labels == c
class_features = support_features[mask]
if len(class_features) > 0:
prototype = np.mean(class_features, axis=0)
prototypes.append(prototype)
else:
prototypes.append(np.zeros_like(support_features[0]))
return np.array(prototypes)
# Test
np.random.seed(42)
sup_feat = np.random.rand(50, 64)
sup_labels = np.array([0]*25 + [1]*25)
protos = compute_prototypes(sup_feat, sup_labels, num_classes=2)
assert protos.shape[0] == 2, "Correct prototype count"
print("✓ Prototype computation working")
if __name__ == "__main__":
print("Lab 1: PrototypeComputation - PASSED")### Lab 2: Prototype Matching
import numpy as np
def prototype_matching(query_features, prototypes, distance_metric='l2'):
"""Match query to nearest prototype"""
distances = []
for q_feat in query_features:
if distance_metric == 'l2':
dists = np.linalg.norm(prototypes - q_feat, axis=1)
else:
dists = -np.dot(prototypes, q_feat)
distances.append(dists)
return np.array(distances)
# Test
np.random.seed(42)
query = np.random.rand(100, 64)
protos = np.random.rand(2, 64)
distances = prototype_matching(query, protos)
assert distances.shape == (100, 2), "Correct distance shape"
print("✓ Prototype matching working")
if __name__ == "__main__":
print("Lab 2: PrototypeMatching - PASSED")### Lab 3: Support Augmentation
import numpy as np
def augment_support_set(support_features, augmentation_factor=3):
"""Augment limited support set"""
augmented = list(support_features)
for feat in support_features:
for _ in range(augmentation_factor):
# Add noise
noise = np.random.randn(*feat.shape) * 0.05
augmented.append(feat + noise)
return np.array(augmented)
# Test
np.random.seed(42)
sup_feat = np.random.rand(5, 64)
aug_feat = augment_support_set(sup_feat, augmentation_factor=2)
assert len(aug_feat) == 15, "Correct augmentation"
print("✓ Support augmentation working")
if __name__ == "__main__":
print("Lab 3: SupportAugmentation - PASSED")### Lab 4: Few-Shot Loss
import numpy as np
def few_shot_segmentation_loss(query_logits, query_labels):
"""Few-shot segmentation loss"""
# Cross-entropy loss
batch_size = query_logits.shape[0]
loss = 0
for b in range(batch_size):
# Log-softmax
log_probs = query_logits[b] - np.max(query_logits[b])
log_probs = log_probs - np.log(np.sum(np.exp(log_probs)))
# Loss for target class
loss -= log_probs[query_labels[b]]
return loss / batch_size
# Test
np.random.seed(42)
logits = np.random.rand(32, 2)
labels = np.array([0] * 16 + [1] * 16)
loss = few_shot_segmentation_loss(logits, labels)
assert np.isfinite(loss), "Loss finite"
print("✓ Few-shot loss working")
if __name__ == "__main__":
print("Lab 4: FewShotLoss - PASSED")