adversarial training

**Adversarial Training (AT)** is the **most effective defense against adversarial attacks** — training the model on adversarial examples by solving a min-max optimization: the inner maximization finds the worst-case perturbation, and the outer minimization trains the model to correctly classify it. **AT Formulation** - **Min-Max**: $min_ heta mathbb{E}_{(x,y)} [max_{|delta| leq epsilon} L(f_ heta(x+delta), y)]$. - **Inner Max**: Use PGD (Projected Gradient Descent) to find the worst-case perturbation $delta^*$. - **Outer Min**: Update model parameters to minimize the loss on the perturbed input $x + delta^*$. - **Epsilon**: The perturbation budget $epsilon$ defines the robustness guarantee. **Why It Matters** - **Gold Standard**: AT remains the most reliable defense against adversarial attacks after years of research. - **PGD-AT**: Madry et al. (2018) showed that PGD adversarial training provides strong empirical robustness. - **Cost**: AT is ~3-10× more expensive than standard training (requires PGD attack at each training step). **Adversarial Training** is **training on the hardest examples** — building robustness by training the model to correctly classify worst-case adversarial perturbations.