Domain Adaptation

What is Domain Adaptation? Techniques to transfer knowledge when source and target domains have different distributions, addressing the "domain shift" problem.

Types of Domain Shift

Domain Adaptation Scenarios

Techniques

Feature Alignment Learn domain-invariant features:

class DomainAdapter(nn.Module):
    def __init__(self, encoder, classifier, discriminator):
        self.encoder = encoder
        self.classifier = classifier
        self.discriminator = discriminator

    def forward(self, source, target):
        source_features = self.encoder(source)
        target_features = self.encoder(target)

        # Classification loss on source
        class_loss = criterion(self.classifier(source_features), labels)

        # Domain confusion loss (adversarial)
        domain_loss = domain_criterion(
            self.discriminator(source_features),
            self.discriminator(target_features)
        )

        return class_loss - lambda_ * domain_loss

Pseudo-Labeling Use model predictions on target domain:

# Generate pseudo-labels
with torch.no_grad():
    target_preds = model(target_data)
    confidence, pseudo_labels = target_preds.max(dim=1)

    # Keep high-confidence predictions
    mask = confidence > threshold

# Train on pseudo-labeled targets
loss = criterion(model(target_data[mask]), pseudo_labels[mask])

Domain Randomization Train on varied source distribution:

# Randomize source domain characteristics
augmented_source = apply_random_transforms(source, {
    "color": True,
    "texture": True,
    "lighting": True
})
# Helps generalize to unseen target domains

Evaluation

Applications

Best Practices

• Analyze source-target distribution gap
• Start with simpler methods (finetuning)
• Use validation split from target domain
• Consider multiple source domains