multi-source domain adaptation
**Multi-source domain adaptation** is a transfer learning approach where knowledge is transferred from **multiple different source domains** simultaneously to improve performance on a target domain. It leverages the diversity of multiple sources to achieve more robust adaptation than single-source approaches.
**Why Multiple Sources Help**
- Different source domains may cover different aspects of the target distribution — together they provide more comprehensive coverage.
- If one source domain is very different from the target, others may be closer — the model can selectively rely on the most relevant sources.
- Multiple perspectives reduce the risk of **negative transfer** from a single poorly matched source.
**Key Challenges**
- **Source Weighting**: Not all sources are equally relevant. The model must learn to weight more relevant sources higher and discount less relevant ones.
- **Domain Conflict**: Sources may conflict with each other — patterns useful in one domain may be harmful for another.
- **Scalability**: Computational cost grows with the number of source domains.
**Methods**
- **Weighted Combination**: Learn weights for each source domain based on its similarity to the target. Sources closer to the target get higher weights.
- **Domain-Specific + Shared Layers**: Use shared representations across all domains plus domain-specific adapter layers for each source.
- **Mixture of Experts**: Each source domain trains a domain-specific expert; a gating network selects which experts to apply for each target example.
- **Domain-Adversarial Multi-Source**: Align each source with the target using separate domain discriminators, then combine aligned features.
- **Moment Matching**: Align the statistical moments (mean, variance, higher-order) of all source and target feature distributions.
**Applications**
- **Sentiment Analysis**: Adapt from reviews in multiple product categories to a new category.
- **Medical Imaging**: Combine data from multiple hospitals (each with different imaging equipment and populations).
- **Autonomous Driving**: Train on data from multiple cities with different driving conditions, adapt to a new city.
- **LLMs**: Pre-training on diverse data sources (books, web, code, Wikipedia) is inherently multi-source.
Multi-source domain adaptation is particularly relevant in the **foundation model era** — large models pre-trained on diverse data naturally embody multi-source transfer.