Value-Based Learning Algorithms

# Multi-Task Learning

## Introduction & Motivation

Multi-Task Learning: train on multiple tasks jointly. Shared representations, auxiliary tasks. Applications: improved generalization, domain knowledge transfer.

Motivation: Leverage task relationships for better learning.

Applications: Computer vision, NLP, robotics.

---

## Core Concepts & Theory

### Shared Encoder

Common feature extractor.

### Task-Specific Heads

Specialized prediction layers.

### Auxiliary Tasks

Supporting objective functions.

### Loss Weighting

Balance multiple objectives.

---

## Mathematical Formulation

Multi-Task Loss: L_{ ext{total}} = \sum_i w_i L_i( heta_{ ext{shared}}, heta_i)

Uncertainty Weighting: L = \sum_i \frac{1}{2\sigma_i^2} L_i + \log \sigma_i

Gradient Normalization: w_i^{(t)} = \frac{\ell_i(t)}{\ell_i(0)} / \sum_j \frac{\ell_j(t)}{\ell_j(0)}

---

## Advanced Theory & Extensions

### Task Affinity

Learn task relationships.

### Adaptive Loss Weighting

Dynamic weight adjustment.

### Modulation Networks

Task-conditional processing.

---

## Computational Considerations

Shared encoder: O(backbone_params).

Task heads: O(|T|·head_params).

Loss computation: O(|T|·batch_size).

---

## Practical Implementation Strategies

### Weight Initialization

Task-specific initialization.

### Loss Scaling

Normalize loss magnitudes.

### Auxiliary Task Selection

Choose supporting tasks.

---

## Benchmark Datasets & Evaluation

Cityscapes: Multi-task scene understanding.

NYU Depth: Depth + semantics.

CelebA: Multi-attribute prediction.

---

## Key Challenges & Limitations

### Task Conflict

Competing gradients.

### Negative Transfer

Auxiliary task interference.

### Hyperparameter Tuning

Weight tuning complexity.

---

## Hyperparameter Tuning

Task weights: 0.1-1.0 per task.

Auxiliary weight: 0.01-0.5.

Shared capacity: 50-75% of total.

---

## Real-World Applications & Case Studies

Autonomous Driving: Detection + segmentation.

Scene Understanding: Depth + semantics + surface normals.

Medical Imaging: Classification + segmentation.

---

## Integration with Other Methods

Multi-task + domain adaptation for universal models; + meta-learning for task discovery.

---

## Summary & Key Takeaways

Multi-Task Learning improves generalization via shared representations.

Principles:
1. Shared encoder: Common features.
2. Task heads: Specialized predictions.
3. Loss weighting: Balance objectives.
4. Auxiliary tasks: Knowledge transfer.
5. Gradient management: Conflict resolution.

---

## Appendix: Practical Labs

### Lab 1: Shared Encoder

import numpy as np

def multitask_forward(x, encoder, heads):
 shared = encoder(x)
 outputs = {}
 for task_name, head in heads.items():
 outputs[task_name] = head(shared)
 return outputs

def simple_encoder(x):
 return x @ np.random.randn(3, 128)

def simple_head(x):
 return x @ np.random.randn(128, 10)

np.random.seed(42)
x = np.random.randn(32, 3, 224, 224).reshape(32, -1)
heads = {'class': simple_head, 'segment': simple_head}
outputs = multitask_forward(x, simple_encoder, heads)
assert len(outputs) == 2, "Correct output count"
print("✓ Shared encoder forward working")

### Lab 2: Uncertainty Weighting

import numpy as np

def uncertainty_weighted_loss(losses, sigmas):
 weighted = 0
 for loss, sigma in zip(losses, sigmas):
 weighted += (1 / (2 * sigma**2)) * loss + np.log(sigma)
 return weighted

np.random.seed(42)
losses = [np.random.rand() for _ in range(3)]
sigmas = [np.random.rand() + 0.5 for _ in range(3)]
total_loss = uncertainty_weighted_loss(losses, sigmas)
assert np.isfinite(total_loss), "Loss is finite"
print("✓ Uncertainty weighting working")

### Lab 3: Gradient Normalization

import numpy as np

def gradient_normalize(losses, prev_losses):
 ratios = np.array([l / p for l, p in zip(losses, prev_losses)])
 weights = ratios / np.sum(ratios)
 return weights

np.random.seed(42)
losses = [0.5, 0.3, 0.2]
prev_losses = [0.6, 0.4, 0.3]
weights = gradient_normalize(losses, prev_losses)
assert np.isclose(weights.sum(), 1), "Weights sum to 1"
print("✓ Gradient normalization working")

### Lab 4: Task Loss Aggregation

import numpy as np

def aggregate_multitask_loss(task_losses, task_weights):
 total = sum(w * l for w, l in zip(task_weights, task_losses))
 return total

np.random.seed(42)
task_losses = [0.5, 0.3, 0.2]
weights = [0.5, 0.3, 0.2]
total_loss = aggregate_multitask_loss(task_losses, weights)
assert total_loss >= 0, "Total loss non-negative"
print("✓ Task loss aggregation working")

---