free adversarial training

**Free Adversarial Training** is a **method that simultaneous updates both the model parameters and the adversarial perturbation in each gradient computation** — reusing the same backward pass for both adversarial example generation and model weight update, making adversarial training essentially "free" in computational cost. **How Free AT Works** - **Shared Gradient**: Compute the gradient $ abla_{x, heta} L(f_ heta(x+delta), y)$ — gradient w.r.t. both input AND parameters. - **Simultaneous Update**: Use the gradient to update $delta$ (for generating adversarial examples) and $ heta$ (for training) in the same step. - **Replay**: Repeat $m$ times on the same minibatch, accumulating perturbation $delta$ across replays. - **Cost**: Total forward-backward passes = $m imes$ standard training (choose $m = 4-8$ for $approx$ PGD-7 robustness). **Why It Matters** - **Computational Free Lunch**: Adversarial perturbation is generated "for free" using the same gradient as weight updates. - **Practical**: Achieves near-PGD-AT robustness at a fraction of the compute cost. - **Memory Efficient**: No need to store separate perturbation gradients — reuses the same computation. **Free AT** is **two-for-one gradient computation** — generating adversarial examples and training the model with a single shared backward pass.