adversarial robustness
**Adversarial Robustness** is the **study of designing and training neural networks that maintain correct predictions when inputs are deliberately perturbed by small, often imperceptible modifications** — addressing the critical vulnerability where state-of-the-art models can be fooled by adding carefully crafted noise that is invisible to humans but causes confident misclassification.
**Adversarial Examples**
- A clean image correctly classified as "panda" → add tiny perturbation (||δ||∞ < 8/255) → model confidently predicts "gibbon".
- Perturbation is imperceptible to humans — image looks identical.
- This is not a rare failure case — it affects every standard neural network.
**Attack Methods**
| Attack | Type | Strength | Method |
|--------|------|----------|--------|
| FGSM | White-box, single-step | Weak | $\delta = \epsilon \cdot sign(\nabla_x L)$ |
| PGD | White-box, iterative | Strong | Multi-step projected gradient descent |
| C&W | White-box, optimization | Very Strong | Minimize perturbation subject to misclassification |
| AutoAttack | Ensemble of attacks | Gold standard | Combination of APGD + targeted attacks |
| Square Attack | Black-box, query-based | Strong | Random search, no gradients needed |
**PGD Attack (Standard Benchmark)**
$x^{t+1} = \Pi_{x+S}(x^t + \alpha \cdot sign(\nabla_x L(f_\theta(x^t), y)))$
- Start from random point within ε-ball around clean input.
- Take multiple gradient ascent steps to maximize loss.
- Project back into ε-ball after each step.
- Typically 20-50 steps with step size α = ε/4.
**Adversarial Training (Primary Defense)**
$\min_\theta E_{(x,y)} [\max_{||\delta||_p \leq \epsilon} L(f_\theta(x + \delta), y)]$
- Inner maximization: Find the worst-case perturbation (using PGD).
- Outer minimization: Update model weights to be correct even on worst-case inputs.
- Cost: 3-10x more expensive than standard training (generating adversarial examples at every step).
- Accuracy trade-off: Robust models typically lose 10-15% clean accuracy.
**Certified Defenses**
- **Randomized Smoothing**: Add Gaussian noise to input → majority vote over noisy predictions.
- Provable guarantee: No perturbation within certified radius can change prediction.
- **IBP (Interval Bound Propagation)**: Compute output bounds for all inputs within ε-ball.
- Trade-off: Certified radius is usually smaller than empirical robustness from adversarial training.
**Robustness Benchmarks**
- **RobustBench**: Standardized leaderboard using AutoAttack on CIFAR-10/ImageNet.
- CIFAR-10 state-of-art: ~70% robust accuracy at ε=8/255 (ℓ∞) — vs. ~98% clean accuracy.
- Gap between clean and robust accuracy highlights the fundamental challenge.
Adversarial robustness is **a critical unsolved problem for deploying AI in safety-sensitive applications** — autonomous vehicles, medical diagnosis, and security systems all require models that cannot be easily deceived, making robustness research essential for trustworthy AI.