gradient masking

**Gradient Masking** is a **phenomenon where a defense accidentally or intentionally makes the model's gradients uninformative** — causing gradient-based attacks to fail while the model remains vulnerable to gradient-free or transfer-based attacks. **Types of Gradient Masking** - **Shattered Gradients**: Non-differentiable operations (JPEG compression, quantization) break gradient flow. - **Stochastic Gradients**: Randomized defenses (random resizing, dropout at inference) make gradients noisy. - **Vanishing/Exploding**: Defenses that cause extreme gradient magnitudes prevent effective optimization. - **Masked Model**: Defensive distillation produces near-zero gradients by softening predictions. **Why It Matters** - **False Security**: Gradient masking makes gradient-based attacks fail, giving the illusion of robustness. - **Transfer Attacks**: Models with masked gradients are still vulnerable to adversarial examples transferred from other models. - **Detection**: If FGSM fails but transfer attacks succeed, gradient masking is likely present. **Gradient Masking** is **hiding the gradient, not fixing the vulnerability** — a defense pitfall that blocks gradient attacks but leaves the model fundamentally exposed.