Self-Ensembling for Domain Adaptation refers to domain adaptation methods that use temporal ensembling or mean teacher techniques—where a slowly-updated copy of the model (teacher) provides pseudo-labels or consistency targets for the current model (student) on unlabeled target data—to achieve domain adaptation without explicit domain alignment losses. Self-ensembling leverages the observation that an exponential moving average (EMA) of model weights produces more stable and accurate predictions than any single checkpoint.

Why Self-Ensembling Matters in AI/ML: Self-ensembling provides domain adaptation without domain alignment, avoiding the adversarial training instability and hyperparameter sensitivity of domain-discriminator methods while achieving competitive or superior performance through simple consistency regularization and pseudo-labeling.

• Mean Teacher framework — The teacher model's weights are an exponential moving average (EMA) of the student's weights: θ_teacher = α · θ_teacher + (1-α) · θ_student, with α typically 0.999; the teacher provides stable predictions on target data that serve as training targets for the student • Consistency loss — The student is trained to produce predictions on target data that are consistent with the teacher's predictions under different augmentations: L_consistency = ||f_student(aug₁(x_T)) - f_teacher(aug₂(x_T))||², encouraging robust representation learning • Confidence-based filtering — Only teacher predictions above a confidence threshold are used as pseudo-labels, filtering out unreliable predictions on hard or ambiguous target samples; this prevents error propagation from incorrect pseudo-labels • No explicit domain alignment — Unlike DANN, MMD, or CORAL methods, self-ensembling does not explicitly minimize domain discrepancy; instead, the combination of source supervision and target consistency implicitly produces domain-invariant features through augmentation-robust learning • Augmentation importance — The effectiveness of self-ensembling depends heavily on the data augmentation strategy: augmentations must be strong enough to create meaningful prediction diversity but not so strong that the teacher's predictions become unreliable

Self-ensembling for domain adaptation elegantly sidesteps explicit domain alignment by instead enforcing prediction consistency between a student model and its slowly-updated teacher copy on augmented target data, achieving competitive domain adaptation through the simple principle that stable, augmentation-invariant predictions naturally produce domain-invariant representations without adversarial training.