Semi-supervised domain adaptation is a transfer learning approach where you have labeled data in the source domain but only limited labeled data (plus unlabeled data) in the target domain. It bridges the gap between fully supervised adaptation (expensive) and unsupervised adaptation (less reliable) by leveraging even a small amount of target labels.

The Setting

• Source Domain: Abundant labeled data (e.g., product reviews from electronics).
• Target Domain: A small number of labeled examples + many unlabeled examples (e.g., product reviews from restaurants).
• Goal: Build a model that performs well on the target domain by combining source knowledge, target labels, and target unlabeled data.

Why It Matters

• In practice, getting some target labels is often feasible — annotating 50–100 examples is practical even when annotating thousands is not.
• A small number of target labels can dramatically improve adaptation quality compared to fully unsupervised approaches.
• It combines the strengths of supervised fine-tuning and unsupervised domain alignment.

Key Methods

• Fine-Tuning with Pseudo-Labels: Fine-tune on limited target labels, then generate pseudo-labels for unlabeled target data using the adapted model. Iterate.
• Domain-Adversarial Training + Target Supervision: Use domain-adversarial networks (DANN) to learn domain-invariant features while also training on the few target labels.
• Consistency Regularization: Require the model to predict the same label for augmented versions of the same unlabeled target example.
• Self-Training: Train on source + target labels, predict on unlabeled target data, add high-confidence predictions to training set, repeat.
• Feature Alignment + Supervised Loss: Align source and target feature distributions while jointly minimizing classification loss on both labeled sets.

Practical Tips

• Active Learning: Strategically select which target examples to label — label the most informative or representative examples rather than random ones.
• Few-Shot Matters: Even 5–10 labeled target examples per class can significantly improve over unsupervised adaptation.

Semi-supervised domain adaptation is the most practical adaptation setting for real-world applications — it reflects the realistic scenario where some labeling effort is possible but large-scale annotation is not.