Adversarial Robustness is the study and engineering of deep learning models that maintain correct predictions when inputs are perturbed by small, carefully crafted adversarial perturbations — imperceptible modifications designed to cause misclassification — encompassing attack methodologies that expose vulnerabilities, empirical defenses that harden models through adversarial training, and certified defenses that provide mathematical guarantees on worst-case performance.
Attack Taxonomy:
- FGSM (Fast Gradient Sign Method): Single-step attack adding epsilon-scaled sign of the loss gradient to the input; fast but relatively weak
- PGD (Projected Gradient Descent): Multi-step iterative attack repeatedly applying small FGSM steps and projecting back onto the epsilon-ball; the standard benchmark attack
- C&W (Carlini & Wagner): Optimization-based attack minimizing perturbation magnitude while ensuring misclassification; effective against many defenses but computationally expensive
- AutoAttack: Ensemble of complementary attacks (APGD-CE, APGD-DLR, FAB, Square) providing a reliable, parameter-free robustness evaluation standard
- Patch Attacks: Modify a localized region (physical sticker, printed pattern) to cause misclassification in real-world settings
- Universal Adversarial Perturbations: Find a single perturbation that fools the model on most inputs, revealing systematic blind spots
Threat Models:
- Lp-Norm Bounded: Perturbations constrained within an Lp ball — L-infinity (max per-pixel change, typically epsilon=8/255 for CIFAR-10), L2 (Euclidean distance), or L1 (sparse perturbations)
- Semantic Perturbations: Physically realizable changes like rotation, color shifts, lighting variations, or weather effects that preserve human interpretation
- Black-Box Attacks: Adversary has no access to model weights; relies on transfer attacks (craft adversarial examples on a surrogate model), query-based attacks, or score-based attacks
- White-Box Attacks: Full access to model architecture, weights, and gradients — the strongest threat model used for rigorous robustness evaluation
Empirical Defenses — Adversarial Training:
- Standard Adversarial Training (Madry et al.): Replace clean training examples with PGD-adversarial examples; the most reliable empirical defense but incurs 3–10x training cost
- TRADES: Decomposes the robust optimization objective into natural accuracy and boundary robustness terms with a tunable tradeoff parameter
- AWP (Adversarial Weight Perturbation): Perturb model weights during adversarial training to flatten the loss landscape and improve generalization
- Friendly Adversarial Training (FAT): Use early-stopped PGD to find adversarial examples near the decision boundary rather than worst-case, reducing overfitting
- Accuracy-Robustness Tradeoff: Adversarially trained models typically sacrifice 5–15% clean accuracy for substantially improved robust accuracy
Certified Defenses:
- Randomized Smoothing: Create a smoothed classifier by averaging predictions over Gaussian noise perturbations of the input; provides L2 certified radii via Neyman-Pearson lemma
- Interval Bound Propagation (IBP): Propagate interval bounds through each network layer to compute guaranteed output bounds for all inputs within the perturbation set
- Linear Relaxation (CROWN, alpha-CROWN): Compute linear upper and lower bounds on network outputs using convex relaxations of nonlinear activations
- Lipschitz Networks: Constrain the Lipschitz constant of each layer (spectral normalization, orthogonal layers) to provably limit output change per unit input perturbation
- Certification Gap: Certified radii are typically smaller than empirical robustness — closing this gap remains an active research challenge
Evaluation Best Practices:
- Use AutoAttack: The standard evaluation suite that prevents overestimating robustness due to gradient masking or obfuscated gradients
- Report Clean and Robust Accuracy: Always measure both natural accuracy and accuracy under attack at the specified epsilon
- Adaptive Attacks: Design attacks specifically targeting the defense mechanism's unique properties; generic attacks may miss exploitable weaknesses
- RobustBench: Standardized benchmark tracking adversarial robustness across models, datasets, and threat models with consistent evaluation protocols
Adversarial robustness remains one of the fundamental open challenges in deploying deep learning to safety-critical domains — where the gap between empirical defenses and provable guarantees, the inherent accuracy-robustness tradeoff, and the computational cost of robust training must all be navigated to build trustworthy AI systems.