pi-model

**Π-Model** (Pi-Model) is a **semi-supervised learning method that enforces consistency between two stochastic forward passes of the same input** — using different dropout masks and/or augmentations for each pass, and penalizing prediction differences. **How Does the Π-Model Work?** - **Two Passes**: Feed the same input $x$ through the network twice with different stochastic noise (dropout, augmentation). - **Consistency Loss**: $mathcal{L}_{cons} = ||f(x, xi_1) - f(x, xi_2)||^2$ where $xi_1, xi_2$ are different noise realizations. - **Total Loss**: $mathcal{L} = mathcal{L}_{CE}( ext{labeled}) + w(t) cdot mathcal{L}_{cons}( ext{all data})$. - **Paper**: Laine & Aila (2017). **Why It Matters** - **Foundation**: One of the earliest and simplest consistency regularization methods. - **Principle**: If the model is good, two noisy views of the same input should give the same prediction. - **Evolution**: Led to Temporal Ensembling → Mean Teacher → MixMatch → FixMatch. **Π-Model** is **the consistency principle distilled** — if a model truly understands an input, it should predict the same thing regardless of noise.