mixup / cutmix

Mixup and CutMix blend training examples to improve model robustness and generalization. **Mixup**: Create virtual training examples by linear interpolation. x̃ = λx₁ + (1-λ)x₂, ỹ = λy₁ + (1-λ)y₂. λ sampled from Beta distribution. Model learns smoother decision boundaries. **CutMix**: Cut and paste image patches between samples, mix labels proportionally to area. Better preserves local features than vanilla mixup. **Why they work**: Regularization effect, encourages linear behavior between training examples, reduces overconfidence, improves calibration. **For NLP**: Mixup in embedding space (hidden layer interpolation), sentence mixing (less common, semantic challenges). **Variants**: Manifold Mixup (mix at hidden layers), CutOut (remove patches, zero labels unchanged), AugMax, Remix. **Training**: Apply with probability p, sample λ per batch, mix within batch. **Results**: 1-3% accuracy improvement on image classification, better out-of-distribution detection. **Hyperparameters**: Alpha for Beta distribution (typically 0.2-0.4), mixing probability. **Implementation**: Simple batch-level operation, minimal overhead. Standard technique for vision model training.