semi-supervised domain adaptation

**Semi-supervised domain adaptation** is a transfer learning approach where you have **labeled data in the source domain** but only **limited labeled data** (plus unlabeled data) in the target domain. It bridges the gap between fully supervised adaptation (expensive) and unsupervised adaptation (less reliable) by leveraging even a small amount of target labels. **The Setting** - **Source Domain**: Abundant labeled data (e.g., product reviews from electronics). - **Target Domain**: A small number of labeled examples + many unlabeled examples (e.g., product reviews from restaurants). - **Goal**: Build a model that performs well on the target domain by combining source knowledge, target labels, and target unlabeled data. **Why It Matters** - In practice, getting **some** target labels is often feasible — annotating 50–100 examples is practical even when annotating thousands is not. - A small number of target labels can dramatically improve adaptation quality compared to fully unsupervised approaches. - It combines the strengths of supervised fine-tuning and unsupervised domain alignment. **Key Methods** - **Fine-Tuning with Pseudo-Labels**: Fine-tune on limited target labels, then generate pseudo-labels for unlabeled target data using the adapted model. Iterate. - **Domain-Adversarial Training + Target Supervision**: Use domain-adversarial networks (DANN) to learn domain-invariant features while also training on the few target labels. - **Consistency Regularization**: Require the model to predict the same label for augmented versions of the same unlabeled target example. - **Self-Training**: Train on source + target labels, predict on unlabeled target data, add high-confidence predictions to training set, repeat. - **Feature Alignment + Supervised Loss**: Align source and target feature distributions while jointly minimizing classification loss on both labeled sets. **Practical Tips** - **Active Learning**: Strategically select which target examples to label — label the most informative or representative examples rather than random ones. - **Few-Shot Matters**: Even **5–10 labeled target examples per class** can significantly improve over unsupervised adaptation. Semi-supervised domain adaptation is the **most practical** adaptation setting for real-world applications — it reflects the realistic scenario where some labeling effort is possible but large-scale annotation is not.