samples per second

**Samples per second** is the **throughput metric measuring how many training examples are processed each second** - it is a core indicator for image and tabular workloads where progress is naturally measured in sample count. **What Is Samples per second?** - **Definition**: Number of individual training samples consumed by the model per second. - **Computation**: Typically derived from global batch size divided by step time, aggregated across workers. - **Sensitivity**: Affected by data loading speed, communication overhead, and kernel efficiency. - **Interpretation Caveat**: Higher throughput is valuable only if model convergence quality remains acceptable. **Why Samples per second Matters** - **Performance Tracking**: Provides immediate signal of system-level throughput improvements or regressions. - **Scaling Analysis**: Helps assess how close distributed training is to linear speedup. - **Cost Efficiency**: More samples per second generally lowers training wall time and infrastructure cost. - **Bottleneck Diagnosis**: Drops often indicate data or communication stalls rather than compute saturation. - **Capacity Planning**: Useful for estimating runtime and cluster demand for new experiments. **How It Is Used in Practice** - **Consistent Measurement**: Report using standardized warm-up handling and averaging windows. - **Pipeline Profiling**: Correlate throughput changes with dataloader and network telemetry. - **Optimization Loop**: Tune batch, prefetch, and parallelism settings while monitoring convergence impact. Samples per second is **a primary throughput KPI for data-parallel training workflows** - when tracked with quality metrics, it drives practical performance optimization.