Mixup and CutMix blend training examples to improve model robustness and generalization. Mixup: Create virtual training examples by linear interpolation. x̃ = λx₁ + (1-λ)x₂, ỹ = λy₁ + (1-λ)y₂. λ sampled from Beta distribution. Model learns smoother decision boundaries. CutMix: Cut and paste image patches between samples, mix labels proportionally to area. Better preserves local features than vanilla mixup. Why they work: Regularization effect, encourages linear behavior between training examples, reduces overconfidence, improves calibration. For NLP: Mixup in embedding space (hidden layer interpolation), sentence mixing (less common, semantic challenges). Variants: Manifold Mixup (mix at hidden layers), CutOut (remove patches, zero labels unchanged), AugMax, Remix. Training: Apply with probability p, sample λ per batch, mix within batch. Results: 1-3% accuracy improvement on image classification, better out-of-distribution detection. Hyperparameters: Alpha for Beta distribution (typically 0.2-0.4), mixing probability. Implementation: Simple batch-level operation, minimal overhead. Standard technique for vision model training.
Explore 500+ Semiconductor & AI Topics
From EUV lithography to CUDA optimization — search the full knowledge base or chat with our AI assistant.