i/o profiling

**I/O profiling** is the **measurement of storage and data-loading throughput from disk to accelerator consumption** - it ensures data supply keeps pace with compute demand and prevents GPU starvation. **What Is I/O profiling?** - **Definition**: Analysis of read bandwidth, latency, queue depth, and preprocessing throughput in input pipelines. - **Data Path**: Storage system, filesystem cache, CPU decode path, and host-to-device transfer stages. - **Key Metrics**: MB per second, sample decode latency, dataloader wait time, and prefetch hit rate. - **Failure Pattern**: Training stalls when model consumption exceeds sustained I/O and preprocessing capacity. **Why I/O profiling Matters** - **Utilization**: Insufficient I/O bandwidth leaves expensive GPUs idle between batches. - **Throughput**: Input pipeline efficiency directly affects samples-per-second and tokens-per-second. - **Scalability**: I/O bottlenecks worsen as cluster size grows without coordinated storage scaling. - **Reliability**: I/O monitoring helps detect filesystem contention and degraded storage nodes early. - **Cost**: Optimized input flow improves compute spend efficiency by increasing productive duty cycle. **How It Is Used in Practice** - **Stage Timing**: Measure each input stage separately to isolate dominant delay contributors. - **Storage Tuning**: Adjust sharding, caching, prefetch depth, and read parallelism based on profile evidence. - **Saturation Check**: Validate that sustained input throughput remains above model consumption across full run. I/O profiling is **a core prerequisite for high-throughput training pipelines** - reliable data delivery must be engineered with the same rigor as model compute.