model flops utilization

**Model FLOPs utilization** is the **efficiency metric that estimates how much useful model computation is achieved relative to hardware capability** - it separates productive model math from overhead and recomputation to provide a more honest training-efficiency view. **What Is Model FLOPs utilization?** - **Definition**: MFU measures effective model-required FLOPs delivered per second divided by hardware peak FLOPs. - **Difference from HFU**: HFU counts all executed operations, while MFU emphasizes useful model work only. - **Penalty Effect**: Activation recomputation and framework overhead lower MFU even if hardware remains busy. - **Use Context**: Widely used in LLM engineering to benchmark end-to-end training stack quality. **Why Model FLOPs utilization Matters** - **Efficiency Honesty**: MFU reveals whether compute cycles are producing model progress or overhead. - **Optimization Priorities**: Helps compare gains from kernel improvements versus algorithmic memory tricks. - **Cross-Run Benchmarking**: Standardized MFU reporting improves transparency across research groups. - **Cost Interpretation**: Higher MFU generally means more useful learning per unit compute spend. - **Architecture Decisions**: MFU trends can guide parallelism and checkpointing strategy choices. **How It Is Used in Practice** - **Metric Definition**: Use consistent model FLOP accounting methodology across experiments. - **Telemetry Pairing**: Track MFU with step time, memory pressure, and communication overhead. - **Optimization Loop**: Tune kernel fusion, overlap strategies, and memory tactics to raise useful compute share. Model FLOPs utilization is **a high-value metric for truthful training efficiency assessment** - it highlights how much hardware effort is converted into actual model learning progress.