Adversarial Training Robustness

# Adversarial Training & Robustness

## Introduction & Motivation

Adversarial training: train on adversarial examples. Improve robustness to attacks. Applications: robust models, security.

Motivation: Defend against adversarial perturbations.

Applications: Secure systems, robust models, trustworthy AI.

---

## Core Concepts & Theory

### Adversarial Examples

Minimal perturbations causing misclassification.

### Robust Loss

Train on worst-case perturbations.

### Fast Adversarial Training

Efficient robust training.

### Certified Defenses

Provable robustness.

---

## Mathematical Formulation

Adversarial Loss:
$$\mathcal{L}_{ ext{adv}} = \max_{\|δ\| ≤ ε} L(x + δ, y)$$

Training Objective:
$$\min_ heta \mathbb{E}_{(x,y)}[\mathcal{L}_{ ext{adv}}]$$

Robustness Radius:
$$r(x) = \arg\max_r \|δ\| ≤ r ext{ s.t. } \hat{y}(x+δ) ≠ y$$

---

## Advanced Theory & Extensions

### Trade-off

Accuracy vs robustness.

### Certified Robustness

Provable radius.

### Randomized Smoothing

Statistical guarantees.

---

## Computational Considerations

Attack generation: O(T·D²).

Training: ~5-10× slower.

Robustness verification: O(n·r).

---

## Practical Implementation Strategies

### Attack Methods

PGD, FGSM variants.

### Perturbation Budgets

Choose epsilon carefully.

### Evaluation

Test on multiple attacks.

---

## Benchmark Datasets & Evaluation

CIFAR-10: Adversarial robustness.

ImageNet: Large-scale robustness.

RobustBench: Benchmark leaderboard.

---

## Key Challenges & Limitations

### Accuracy Drop

Robust models less accurate.

### Computational Cost

Much slower training.

### Transferability

Attacks may not transfer.

---

## Hyperparameter Tuning

Epsilon: 8/255-255/255.

Step size: 2/255.

Steps: 10-20.

---

## Real-World Applications & Case Studies

Autonomous Driving: Robust to adversarial attacks.

Biometric Security: Prevent spoofing.

Fraud Detection: Detect adversarial attempts.

---

## Integration with Other Methods

Adversarial training + certified defenses; + ensemble methods.

---

## Summary & Key Takeaways

Adversarial training improves model robustness.

Principles:
1. Adversarial examples: Worst-case perturbations.
2. Robust training: Learn from adversarial.
3. Defense: Improve robustness.
4. Evaluation: Test thoroughly.
5. Trade-off: Balance accuracy and robustness.

---

## Appendix: Practical Labs

### Lab 1: FGSM Attack

import numpy as np

def fgsm_attack(model, x, y, epsilon=0.03):
 """Fast Gradient Sign Method attack"""
 # Compute gradient
 grad = np.random.randn(*x.shape) # Simplified
 
 # Apply perturbation
 x_adv = x + epsilon * np.sign(grad)
 x_adv = np.clip(x_adv, 0, 1)
 
 return x_adv

np.random.seed(42)
x = np.random.rand(224, 224, 3)
y = 1
x_adv = fgsm_attack(None, x, y)
assert x_adv.shape == x.shape
print("✓ FGSM attack working")

### Lab 2: PGD Attack

import numpy as np

def pgd_attack(model, x, y, epsilon=0.03, steps=10):
 """Projected Gradient Descent attack"""
 x_adv = x.copy()
 
 for _ in range(steps):
 grad = np.random.randn(*x.shape) # Simplified
 x_adv = x_adv + (epsilon / steps) * np.sign(grad)
 x_adv = np.clip(x_adv, x - epsilon, x + epsilon)
 x_adv = np.clip(x_adv, 0, 1)
 
 return x_adv

np.random.seed(42)
x = np.random.rand(10)
y = 1
x_adv = pgd_attack(None, x, y)
assert np.linalg.norm(x_adv - x) <= 0.03 * 1.5
print("✓ PGD attack working")

### Lab 3: Robustness Evaluation

import numpy as np

def evaluate_robustness(model, test_data, epsilons):
 """Evaluate model robustness across epsilons"""
 results = []
 
 for epsilon in epsilons:
 # Attack and evaluate
 robust_acc = np.random.rand() # Simplified
 results.append(robust_acc)
 
 return results

epsilons = [0, 0.01, 0.03, 0.05]
robustness = evaluate_robustness(None, [], epsilons)
assert len(robustness) == len(epsilons)
print("✓ Robustness evaluation working")

### Lab 4: Certified Radius

import numpy as np

def certified_radius(predictions, num_classes, sigma=0.25):
 """Compute certified robustness radius"""
 top_2 = np.argsort(predictions)[-2:]
 top_confidence = predictions[top_2[1]]
 
 radius = (sigma / 2) * (np.arccos(2 * top_confidence - 1))
 return radius

predictions = np.random.rand(1000)
radius = certified_radius(predictions)
assert radius >= 0
print(f"✓ Certified radius: {radius:.4f}")

---