yopo

**YOPO** (You Only Propagate Once) is a **fast adversarial training method based on the observation that adversarial perturbations mainly depend on the first layer's gradients** — by restricting full backpropagation to the first layer and updating the perturbation with cheap first-layer gradient computations. **How YOPO Works** - **Key Insight**: The adversarial perturbation $delta$ is an input-space quantity — its gradient primarily depends on the first layer. - **Full Backprop**: Perform one full forward-backward pass to update model weights. - **Cheap Updates**: Perform $p$ additional cheap perturbation updates using only the first layer's gradient. - **Cost Reduction**: Full backprop once + $p$ cheap first-layer passes ≈ $1 + p cdot epsilon$ forward-backward cost (where $epsilon ll 1$). **Why It Matters** - **Theoretical Foundation**: Based on the Pontryagin's Maximum Principle (PMP) connection to adversarial training. - **Efficiency**: Achieves PGD-level robustness with significantly fewer full backward passes. - **Scalable**: The first-layer gradient computation is much cheaper than full backpropagation. **YOPO** is **cheap perturbation updates** — exploiting the structure of adversarial perturbations to avoid repeated full backpropagation.