Data Augmentation Strategies

# Data Augmentation Strategies

## Introduction & Motivation

Data Augmentation: expand training data synthetically. Image augmentation, text augmentation. Applications: improve generalization, handle imbalance, reduce overfitting.

Motivation: Enhance dataset diversity without manual labeling.

Applications: Image classification, NLP, semi-supervised learning.

---

## Core Concepts & Theory

### Image Augmentation

Rotation, flip, crop, color jitter.

### Text Augmentation

Paraphrasing, back-translation, token shuffling.

### Mixup

Linear interpolation of samples.

### CutMix

Spatial augmentation mixing.

---

## Mathematical Formulation

Mixup:
$$ ilde{x} = \lambda x_i + (1-\lambda) x_j, \quad ilde{y} = \lambda y_i + (1-\lambda) y_j$$

CutMix:
$$ ilde{x} = x_i \odot M + x_j \odot (1-M)$$

Augmentation Pipeline:
$$x_{ ext{aug}} = ext{compose}(f_1, f_2, ..., f_k)(x)$$

---

## Advanced Theory & Extensions

### AutoAugment

Learned augmentation policies.

### Mixup Variants

Manifold mixup, cutmix evolution.

### Temporal Augmentation

Video/sequence augmentation.

---

## Computational Considerations

Image ops: O(H·W·C).

Text ops: O(tokens).

Mixup: O(batch_size).

---

## Practical Implementation Strategies

### Pipeline Composition

Chain multiple augmentations.

### Probability Control

Apply augmentations probabilistically.

### Strength Tuning

Control augmentation intensity.

---

## Benchmark Datasets & Evaluation

CIFAR-10: Augmentation benchmark.

ImageNet: Large-scale validation.

SST-2: Text augmentation evaluation.

---

## Key Challenges & Limitations

### Augmentation-Label Mismatch

Invalid synthetic examples.

### Computational Overhead

Augmentation time cost.

### Domain Specificity

Augmentation tuning per domain.

---

## Hyperparameter Tuning

Mixup alpha: 0.1-1.0.

Augmentation probability: 0.5-0.9.

Rotation angle: 10-30 degrees.

---

## Real-World Applications & Case Studies

Medical Imaging: Limited data handling.

Autonomous Driving: Simulation augmentation.

Few-Shot Learning: Data expansion.

---

## Integration with Other Methods

Augmentation + semi-supervised learning for pseudo-labeling; + adversarial training for robustness.

---

## Summary & Key Takeaways

Data Augmentation improves model robustness via synthetic data expansion.

Principles:
1. Image transforms: Rotation, flips, crops.
2. Text transforms: Paraphrase, back-translation.
3. Mixup: Convex combination.
4. CutMix: Spatial mixing.
5. Policy learning: AutoAugment strategies.

---

## Appendix: Practical Labs

### Lab 1: Mixup Implementation

import numpy as np

def mixup(x1, y1, x2, y2, alpha=0.2):
 """Apply mixup augmentation"""
 lam = np.random.beta(alpha, alpha)
 x_mixed = lam * x1 + (1 - lam) * x2
 y_mixed = lam * y1 + (1 - lam) * y2
 return x_mixed, y_mixed

np.random.seed(42)
x1, y1 = np.random.randn(3, 224, 224), 0
x2, y2 = np.random.randn(3, 224, 224), 1
x_mix, y_mix = mixup(x1, y1, x2, y2)
assert x_mix.shape == x1.shape, "Correct mixed shape"
assert 0 <= y_mix <= 1, "Mixed label in valid range"
print("✓ Mixup working")

### Lab 2: CutMix Implementation

import numpy as np

def cutmix(x1, y1, x2, y2, alpha=1.0):
 """Apply cutmix augmentation"""
 lam = np.random.beta(alpha, alpha)
 h, w = x1.shape[1:3]
 cut_ratio = np.sqrt(1 - lam)
 cut_h = int(w * cut_ratio)
 cut_w = int(h * cut_ratio)
 
 cx = np.random.randint(0, w)
 cy = np.random.randint(0, h)
 x_mixed = x1.copy()
 x_mixed[:, cy:cy+cut_h, cx:cx+cut_w] = x2[:, cy:cy+cut_h, cx:cx+cut_w]
 
 return x_mixed, lam * y1 + (1 - lam) * y2

np.random.seed(42)
x1 = np.random.randn(3, 32, 32)
x2 = np.random.randn(3, 32, 32)
x_mix, y_mix = cutmix(x1, 0, x2, 1)
assert x_mix.shape == x1.shape, "Correct mixed shape"
print("✓ CutMix working")

### Lab 3: Random Rotation

import numpy as np

def random_rotation(image, angle_range=30):
 """Apply random rotation"""
 angle = np.random.uniform(-angle_range, angle_range)
 h, w = image.shape[:2]
 center = (w // 2, h // 2)
 
 rotated = np.rot90(image, k=1) if angle > 0 else image
 return rotated

np.random.seed(42)
image = np.random.rand(32, 32, 3)
rotated = random_rotation(image)
assert rotated.shape == image.shape, "Correct rotated shape"
print("✓ Random rotation working")

### Lab 4: Color Jitter

import numpy as np

def color_jitter(image, brightness=0.2, contrast=0.2):
 """Apply color jitter augmentation"""
 jittered = image.copy()
 
 b_factor = np.random.uniform(1 - brightness, 1 + brightness)
 jittered = jittered * b_factor
 
 c_factor = np.random.uniform(1 - contrast, 1 + contrast)
 jittered = (jittered - 0.5) * c_factor + 0.5
 
 return np.clip(jittered, 0, 1)

np.random.seed(42)
image = np.random.rand(32, 32, 3)
jittered = color_jitter(image)
assert jittered.shape == image.shape, "Correct jittered shape"
assert np.all(jittered >= 0) and np.all(jittered <= 1), "Values in valid range"
print("✓ Color jitter working")

---