robust training methods

**Robust Training Methods** are **training algorithms that produce neural networks resilient to adversarial perturbations, noise, and distribution shift** — going beyond standard ERM (Empirical Risk Minimization) to explicitly optimize for worst-case or perturbed-case performance. **Key Robust Training Approaches** - **Adversarial Training (AT)**: Train on adversarial examples generated during training (PGD-AT). - **TRADES**: Trade off clean accuracy and robustness with an explicit regularization term. - **Certified Training**: Train to maximize certified robustness radius (IBP training, CROWN-IBP). - **Data Augmentation**: Heavy augmentation (AugMax, adversarial augmentation) improves distributional robustness. **Why It Matters** - **Standard Training Fails**: Standard ERM produces models that are trivially fooled by small perturbations. - **Defense**: Robust training is the most effective defense against adversarial attacks — far better than post-hoc defenses. - **Trade-Off**: Robust models typically sacrifice some clean accuracy for improved worst-case performance. **Robust Training** is **training for the worst case** — explicitly optimizing models to maintain performance under adversarial and noisy conditions.