virtual adversarial training

**VAT** (Virtual Adversarial Training) is a **semi-supervised regularization technique that computes the worst-case perturbation to inputs and penalizes the model for changing its predictions** — enforcing local smoothness of the output distribution around both labeled and unlabeled data. **How Does VAT Work?** - **Find Adversarial Direction**: $r_{adv} = argmax_{||r|| leq epsilon} ext{KL}(p(y|x) || p(y|x+r))$ (direction that maximally changes predictions). - **Power Iteration**: Approximate $r_{adv}$ using 1-2 steps of power iteration (efficient). - **Loss**: $mathcal{L}_{VAT} = ext{KL}(p(y|x) || p(y|x+r_{adv}))$ (penalize prediction change under worst-case perturbation). - **Paper**: Miyato et al. (2018). **Why It Matters** - **No Labels Needed**: The VAT loss is computed without labels -> can be applied to unlabeled data. - **Local Smoothness**: Enforces that predictions are robust to small input perturbations. - **Universal**: Works for any model differentiable with respect to its input (images, text embeddings, etc.). **VAT** is **adversarial robustness as regularization** — finding and defending against worst-case perturbations to enforce smooth, confident predictions.