TRADES (TRadeoff-inspired Adversarial DEfense via Surrogate-loss minimization) is a robust training method that explicitly balances clean accuracy and adversarial robustness — decomposing the robust risk into natural error plus a boundary error regularization term.

TRADES Formulation

• Objective: $min_ heta mathbb{E}[underbrace{L(f(x), y)}_{ ext{natural loss}} + eta underbrace{max_{|delta|leqepsilon} KL(f(x) | f(x+delta))}_{ ext{robustness regularizer}}]$.
• Natural Loss: Standard cross-entropy on clean inputs (maintains clean accuracy).
• Robustness Term: KL divergence between clean and adversarial predictions (encourages consistent predictions).
• Trade-Off ($eta$): Higher $eta$ = more robust but lower clean accuracy. Lower $eta$ = higher clean accuracy but less robust.

Why It Matters

• Better Trade-Off: TRADES achieves better accuracy-robustness trade-offs than standard adversarial training.
• Theoretical Foundation: Grounded in the decomposition of robust risk (Zhang et al., 2019).
• Tunable: The $eta$ parameter gives explicit control over the accuracy-robustness trade-off.

TRADES is the balanced defense — explicitly optimizing both clean accuracy and adversarial robustness with a tunable trade-off parameter.