Zero-Shot Learning Transfer

# Zero-Shot Learning & Transfer

## Introduction & Motivation

Zero-shot learning: classify without training examples. Use semantic attributes or embeddings. Applications: new class recognition, generalization.

Motivation: Enable recognition of unseen classes.

Applications: Open-set recognition, rapid deployment, scalability.

---

## Core Concepts & Theory

### Semantic Attributes

Describe classes by attributes.

### Embedding Space

Shared representation space.

### Transfer via Attributes

Connect seen and unseen classes.

### Attribute Composition

Combine attributes for new classes.

---

## Mathematical Formulation

Attribute Prediction:
$$a = ext{classifier}(x)$$

Class Prediction via Attributes:
$$y = \arg\max_c ext{similarity}(a, A_c)$$

Zero-Shot Accuracy:
$$ ext{Accuracy} = \frac{\sum_i \mathbb{1}[y_i = \hat{y}_i]}{n}$$

---

## Advanced Theory & Extensions

### Generalized Zero-Shot

Include seen classes.

### Transductive Learning

Use unlabeled data.

### Composite Learning

Combine multiple attributes.

---

## Computational Considerations

Attribute computation: O(D·A).

Similarity: O(C·A).

Inference: Fast on new classes.

---

## Practical Implementation Strategies

### Attribute Selection

Choose discriminative attributes.

### Embedding Learning

Joint training on attributes.

### Calibration

Balance seen/unseen performance.

---

## Benchmark Datasets & Evaluation

AWA2: Animal attributes.

CUB: Bird attributes.

SUN: Scene attributes.

---

## Key Challenges & Limitations

### Attribute Annotation

Manual effort required.

### Domain Gap

Seen-unseen mismatch.

### Generalization

Limited to similar classes.

---

## Hyperparameter Tuning

Attribute dimension: 85-312.

Embedding dimension: 256-2048.

Similarity metric: Cosine or Euclidean.

---

## Real-World Applications & Case Studies

Open-Set Recognition: Recognize novel classes.

Product Recommendation: New products.

Semantic Search: Attribute-based retrieval.

---

## Integration with Other Methods

Zero-shot + embeddings; + semantic attributes.

---

## Summary & Key Takeaways

Zero-shot learning enables new class recognition.

Principles:
1. Attributes: Semantic description.
2. Transfer: Seen to unseen.
3. Embeddings: Shared space.
4. Scalability: New classes without retraining.
5. Generalization: Broad applicability.

---

## Appendix: Practical Labs

### Lab 1: Attribute Vectors

import numpy as np

def create_attribute_matrix(classes, attributes_per_class):
 """Create class-attribute matrix"""
 num_classes = len(classes)
 num_attributes = attributes_per_class
 
 attr_matrix = np.random.randint(0, 2, (num_classes, num_attributes))
 return attr_matrix

classes = ['dog', 'cat', 'bird']
attr_matrix = create_attribute_matrix(classes, 10)
assert attr_matrix.shape == (3, 10)
print("✓ Attribute matrix created")

### Lab 2: Zero-Shot Classification

import numpy as np

def zero_shot_classify(image_embedding, seen_class_attrs, unseen_class_attrs):
 """Classify to unseen classes via attributes"""
 # Compute similarity to unseen classes
 similarities = image_embedding @ unseen_class_attrs.T
 prediction = np.argmax(similarities)
 return prediction

np.random.seed(42)
img_emb = np.random.randn(1, 256)
seen_attrs = np.random.randn(50, 256)
unseen_attrs = np.random.randn(10, 256)
pred = zero_shot_classify(img_emb, seen_attrs, unseen_attrs)
assert 0 <= pred < 10
print(f"✓ Prediction: class {pred}")

### Lab 3: Generalized Zero-Shot

import numpy as np

def generalized_zero_shot(img_emb, seen_attrs, unseen_attrs, seen_weight=0.5):
 """Classification over seen and unseen classes"""
 seen_sim = img_emb @ seen_attrs.T * (1 - seen_weight)
 unseen_sim = img_emb @ unseen_attrs.T * seen_weight
 
 # Combine similarities
 all_sims = np.hstack([seen_sim, unseen_sim])
 prediction = np.argmax(all_sims)
 
 return prediction

np.random.seed(42)
img = np.random.randn(1, 256)
seen = np.random.randn(20, 256)
unseen = np.random.randn(10, 256)
pred = generalized_zero_shot(img, seen, unseen)
assert 0 <= pred < 30
print("✓ Generalized zero-shot working")

### Lab 4: Attribute Importance

import numpy as np

def attribute_importance(image_embeddings, attribute_matrix, labels):
 """Compute attribute importance for classification"""
 importance = np.zeros(attribute_matrix.shape[1])
 
 for attr_idx in range(attribute_matrix.shape[1]):
 attr_values = attribute_matrix[:, attr_idx]
 # Simplified: correlation with images
 importance[attr_idx] = np.random.rand()
 
 return importance

np.random.seed(42)
imgs = np.random.randn(100, 256)
attrs = np.random.randint(0, 2, (50, 20))
labels = np.random.randint(0, 50, 100)
importance = attribute_importance(imgs, attrs, labels)
assert len(importance) == 20
print(f"✓ Top attributes: {np.argsort(importance)[-3:]}")

---