Mean Teacher is a semi-supervised learning method that maintains an exponential moving average (EMA) of model weights as the "teacher" — the student model is trained on labeled data, while a consistency loss encourages the student to match the teacher's predictions on unlabeled data.

How Does Mean Teacher Work?

• Student: Trained with gradient descent on labeled loss + consistency loss.
• Teacher: $ heta_{teacher} = alpha cdot heta_{teacher} + (1-alpha) cdot heta_{student}$ (EMA of student weights, $alpha approx 0.999$).
• Consistency: $mathcal{L}_{cons} = ||f_{teacher}(x + ext{noise}) - f_{student}(x + ext{noise'})||^2$.
• Paper: Tarvainen & Valpola (2017).

Why It Matters

• Stable Targets: The EMA teacher produces more stable and accurate targets than the student alone.
• Foundation: Inspired BYOL (self-supervised), EMA-based methods in detection, and modern SSL frameworks.
• Effective: Significant improvement over Temporal Ensembling with negligible additional compute.

Mean Teacher is the smoothed mentor — an exponentially averaged version of the model that provides stable, high-quality targets for semi-supervised learning.