Domain generalization (DG) trains machine learning models to perform well on entirely unseen target domains without any access to target domain data during training. Unlike domain adaptation (which accesses unlabeled target data), DG must learn representations robust enough to handle arbitrary domain shifts.

Why Domain Generalization Matters

• Unknown Deployment: In real-world applications, you often cannot anticipate what domain shift the model will face. A medical model trained on Hospital A's scanners must work on Hospital B's different equipment.
• No Target Access: Collecting even unlabeled data from every possible target domain is impractical — there are too many potential deployment environments.
• Safety Critical: Autonomous driving models must handle unseen weather conditions, cities, and lighting without failure.

Techniques

• Invariant Risk Minimization (IRM): Learn features whose predictive relationships are consistent across all training domains. If feature X predicts label Y in Domain 1 but not Domain 2, discard feature X.
• Domain-Invariant Representation Learning: Use adversarial training or MMD (Maximum Mean Discrepancy) to align feature distributions across source domains. If the model can't distinguish which domain an embedding came from, the features are domain-invariant.
• Data Augmentation for Domain Shift: Simulate unseen domains through:
• Style Transfer: Apply random artistic styles to training images.
• Random Convolution: Apply randomly initialized convolution filters as data augmentation.
• Frequency Domain Perturbation: Swap low-frequency components (style) between images.
• MixStyle: Interpolate feature statistics between different domain samples.

• Meta-Learning for DG: Simulate train-test domain shift during training by holding out one source domain for validation in each episode. Forces the model to learn features that generalize to the held-out domain.
• MLDG (Meta-Learning Domain Generalization): MAML-inspired approach that optimizes for cross-domain transfer.

• Causal Learning: Learn causal features (genuinely predictive relationships) rather than spurious correlations (domain-specific shortcuts). Causal relationships remain stable across domains.

Benchmark Datasets

Evaluation Protocol

• Leave-One-Domain-Out: Train on all source domains except one, test on the held-out domain. Repeat for each domain.
• Training-Domain Validation: Use data from training domains only for model selection — no peeking at the target.

Key Findings

• ERM is Surprisingly Strong: Simple Empirical Risk Minimization (standard training) with modern architectures often matches or beats complex DG methods (Gulrajani & Lopez-Paz, 2021).
• Foundation Models Excel: Large pre-trained models (CLIP, DINOv2) show strong domain generalization naturally, likely because they've seen diverse domains during pre-training.
• Diverse Pre-Training > Algorithms: Training on more diverse data seems more effective than sophisticated DG algorithms.

Domain generalization remains an open research challenge — the gap between in-domain and out-of-domain performance persists, and no method reliably generalizes across all types of domain shifts.