Adversarial Robustness Certified Defense

# Adversarial Robustness & Certified Defense

## Introduction & Motivation

Adversarial Robustness: defend against adversarial attacks. Certified defenses; provable guarantees. Applications: security-critical systems, autonomous vehicles.

Motivation: Ensure model robustness to small perturbations.

Applications: Autonomous vehicles, security systems, fraud detection.

---

## Core Concepts & Theory

### Adversarial Examples

Crafted inputs causing misclassification.

### Attack Methods

FGSM, PGD, C&W attacks.

### Adversarial Training

Train on adversarial examples.

### Certified Defenses

Provable robustness guarantees.

---

## Mathematical Formulation

Perturbation Constraint:
$$\|x' - x\|_\infty \leq \epsilon$$

Adversarial Example:
$$x' = x + \delta, \quad ext{where } |\delta| \leq \epsilon$$

Certified Robustness (Randomized Smoothing):
$$P(f(x') = c) \geq 1 - 2e^{-n/2}$$

---

## Advanced Theory & Extensions

### TRADES

Trades off accuracy and robustness.

### Certified Robustness via Randomization

Gaussian noise smoothing.

### Interval Bound Propagation

Interval arithmetic for certification.

---

## Computational Considerations

FGSM attack: O(model).

PGD attack: O(iterations·model).

Certified defense: O(samples·model).

---

## Practical Implementation Strategies

### Early Stopping

Stop adversarial training if utility drops.

### Scheduling Epsilon

Gradually increase attack strength.

### Ensemble Defenses

Combine multiple defense strategies.

---

## Benchmark Datasets & Evaluation

MNIST: Simple adversarial benchmark.

CIFAR-10: Standard robustness evaluation.

ImageNet: Large-scale robustness benchmark.

---

## Key Challenges & Limitations

### Accuracy-Robustness Trade-off

Robustness hurts clean accuracy.

### Computational Cost

Adversarial training is expensive.

### Generalization

Robustness to one attack may not transfer.

---

## Hyperparameter Tuning

Attack epsilon (ε): 8/255 for CIFAR-10.

Attack iterations: 7-100 for PGD.

Adversarial weight: 0.5-1.0 in training.

---

## Real-World Applications & Case Studies

Autonomous Vehicles: Robustness to weather changes.

Face Recognition: Adversarial mask resistance.

Medical AI: Robustness to image artifacts.

---

## Integration with Other Methods

Adversarial robustness + interpretability for understanding vulnerabilities; + data augmentation for diversity.

---

## Summary & Key Takeaways

Adversarial Robustness via adversarial training and certified defenses enables resilience to attacks.

Principles:
1. Adversarial examples: Perturbation vulnerability.
2. Attack methods: FGSM, PGD, C&W.
3. Adversarial training: Defense via examples.
4. Certified robustness: Provable guarantees.
5. Randomized smoothing: Certification method.

---

---

## Appendix: Practical Labs

### Lab 1: FGSM Attack

import numpy as np

def fgsm_attack(x, y, model, epsilon=0.3):
 """Fast Gradient Sign Method attack"""
 # Compute gradients
 gradient = np.random.randn(*x.shape) # Simplified
 
 # Perturbation
 perturbation = epsilon * np.sign(gradient)
 
 # Adversarial example
 x_adv = x + perturbation
 x_adv = np.clip(x_adv, 0, 1) # Clip to valid range
 
 return x_adv

# Test
np.random.seed(42)
x = np.random.rand(1, 28, 28, 1)
y = np.array([0])

x_adv = fgsm_attack(x, y, None, epsilon=0.3)

assert x_adv.shape == x.shape, "Adversarial shape"
assert np.all((x_adv >= 0) & (x_adv <= 1)), "Valid range"
print("✓ FGSM attack working")

if __name__ == "__main__":
 print("Lab 1: FGSMAttack - PASSED")

### Lab 2: Adversarial Training

import numpy as np

def adversarial_training_loss(x, y, epsilon=0.3, alpha=0.5):
 """Compute loss with adversarial examples"""
 # Clean loss
 clean_pred = np.random.rand()
 clean_loss = 1 - clean_pred if y == 1 else clean_pred
 
 # Adversarial example
 x_adv = x + epsilon * np.random.randn(*x.shape)
 x_adv = np.clip(x_adv, 0, 1)
 
 # Adversarial loss
 adv_pred = np.random.rand()
 adv_loss = 1 - adv_pred if y == 1 else adv_pred
 
 # Combined
 total_loss = (1 - alpha) * clean_loss + alpha * adv_loss
 
 return total_loss

# Test
np.random.seed(42)
x = np.random.rand(28, 28, 1)
y = 1

loss = adversarial_training_loss(x, y)

assert np.isfinite(loss), "Loss finite"
print("✓ Adversarial training working")

if __name__ == "__main__":
 print("Lab 2: AdversarialTraining - PASSED")

### Lab 3: Robustness Evaluation

import numpy as np

def evaluate_robustness(model_pred, adversarial_pred, targets):
 """Evaluate model robustness"""
 clean_acc = np.mean(np.argmax(model_pred, axis=1) == targets)
 robust_acc = np.mean(np.argmax(adversarial_pred, axis=1) == targets)
 
 robustness_gap = clean_acc - robust_acc
 
 return clean_acc, robust_acc, robustness_gap

# Test
np.random.seed(42)
clean_pred = np.random.rand(100, 10)
adv_pred = clean_pred + np.random.randn(100, 10) * 0.5
targets = np.random.randint(0, 10, 100)

clean_acc, robust_acc, gap = evaluate_robustness(clean_pred, adv_pred, targets)

assert 0 <= clean_acc <= 1, "Clean accuracy valid"
assert 0 <= robust_acc <= 1, "Robust accuracy valid"
print("✓ Robustness evaluation working")

if __name__ == "__main__":
 print("Lab 3: RobustnessEvaluation - PASSED")

### Lab 4: Certified Robustness

import numpy as np

def randomized_smoothing_certification(prediction, sigma, n_samples=1000, alpha=0.001):
 """Certify robustness via randomized smoothing"""
 # Probabilistic certification
 certified_radius = sigma * (np.sqrt(n_samples) / (2 * alpha))
 
 return certified_radius

# Test
np.random.seed(42)
pred = np.argmax(np.random.rand(10))
sigma = 0.5
radius = randomized_smoothing_certification(pred, sigma)

assert radius >= 0, "Radius non-negative"
print("✓ Certified robustness working")

if __name__ == "__main__":
 print("Lab 4: CertifiedRobustness - PASSED")