domain adaptation
**Domain Adaptation**
**What is Domain Adaptation?**
Techniques to transfer knowledge when source and target domains have different distributions, addressing the "domain shift" problem.
**Types of Domain Shift**
| Shift Type | Example |
|------------|---------|
| Covariate | Different input distributions |
| Label | Different class distributions |
| Concept | Same input, different meaning |
| Prior | Different class frequencies |
**Domain Adaptation Scenarios**
| Scenario | Source Labels | Target Labels |
|----------|---------------|---------------|
| Supervised | Yes | Yes |
| Semi-supervised | Yes | Few |
| Unsupervised | Yes | No |
**Techniques**
**Feature Alignment**
Learn domain-invariant features:
```python
class DomainAdapter(nn.Module):
def __init__(self, encoder, classifier, discriminator):
self.encoder = encoder
self.classifier = classifier
self.discriminator = discriminator
def forward(self, source, target):
source_features = self.encoder(source)
target_features = self.encoder(target)
# Classification loss on source
class_loss = criterion(self.classifier(source_features), labels)
# Domain confusion loss (adversarial)
domain_loss = domain_criterion(
self.discriminator(source_features),
self.discriminator(target_features)
)
return class_loss - lambda_ * domain_loss
```
**Pseudo-Labeling**
Use model predictions on target domain:
```python
# Generate pseudo-labels
with torch.no_grad():
target_preds = model(target_data)
confidence, pseudo_labels = target_preds.max(dim=1)
# Keep high-confidence predictions
mask = confidence > threshold
# Train on pseudo-labeled targets
loss = criterion(model(target_data[mask]), pseudo_labels[mask])
```
**Domain Randomization**
Train on varied source distribution:
```python
# Randomize source domain characteristics
augmented_source = apply_random_transforms(source, {
"color": True,
"texture": True,
"lighting": True
})
# Helps generalize to unseen target domains
```
**Evaluation**
| Metric | Description |
|--------|-------------|
| Target accuracy | Performance on target |
| Source accuracy | Maintain source performance |
| Domain gap | Measure distribution difference |
**Applications**
| Domain | Example |
|--------|---------|
| Vision | Synthetic to real images |
| NLP | Formal to informal text |
| Medical | Hospital A to Hospital B |
| Robotics | Simulation to real robot |
**Best Practices**
- Analyze source-target distribution gap
- Start with simpler methods (finetuning)
- Use validation split from target domain
- Consider multiple source domains