fast adversarial training

**Fast Adversarial Training** is a **computationally efficient variant of adversarial training that uses single-step attacks (FGSM) instead of multi-step PGD** — reducing the training cost from ~10× standard training (PGD-AT) to ~2× while maintaining competitive robustness. **How Fast AT Works** - **FGSM + Random Init**: Use FGSM with random initialization instead of multi-step PGD. - **Single Step**: Only one gradient computation per adversarial example (vs. 7-20 for PGD). - **Catastrophic Overfitting**: Na ̈ive FGSM-AT can suffer from catastrophic overfitting — robustness suddenly drops to 0%. - **Fixes**: Random initialization, gradient regularization (GradAlign), and early stopping prevent catastrophic overfitting. **Why It Matters** - **Speed**: ~5× faster than PGD-AT — makes adversarial training practical for large models. - **Accessibility**: Enables adversarial training on limited compute budgets. - **Surprising Effectiveness**: With proper initialization, single-step FGSM-AT achieves ~90% of PGD-AT robustness. **Fast AT** is **adversarial training on a budget** — using single-step attacks for efficient robust training with proper safeguards against catastrophic overfitting.