balanced sampling

**Balanced Sampling** is a **data loading strategy that constructs mini-batches with equal (or balanced) representation of each class** — ensuring every class appears proportionally in each training batch, regardless of the original class distribution in the dataset. **Balanced Sampling Strategies** - **Class-Balanced**: Sample equal numbers from each class per batch — each batch has $B/C$ samples per class. - **Square-Root Sampling**: Sample proportional to $sqrt{n_c}$ — a compromise between balanced and natural frequency. - **Progressively Balanced**: Start with natural frequency, gradually shift to balanced sampling during training. - **Instance-Balanced**: Sample all instances equally, ensuring rare instances get represented. **Why It Matters** - **Mini-Batch Coverage**: With natural sampling, rare classes may not appear in many mini-batches — balanced sampling ensures coverage. - **Gradient Diversity**: Balanced batches provide gradient updates from all classes — better optimization landscape. - **Trade-Off**: Fully balanced sampling over-represents rare classes — can cause overfitting on minority classes. **Balanced Sampling** is **equal airtime for all classes** — constructing training batches with proportional class representation regardless of dataset imbalance.