flops

FLOPS (floating point operations per second) measures a processor's computational throughput, serving as the primary metric for comparing AI hardware capabilities and estimating training/inference requirements. Units: (1) TFLOPS—teraFLOPS (10¹² ops/sec), typical for single GPU; (2) PFLOPS—petaFLOPS (10¹⁵), typical for GPU clusters; (3) EFLOPS—exaFLOPS (10¹⁸), frontier supercomputers. GPU FLOPS by generation (NVIDIA, FP16/BF16): (1) V100—125 TFLOPS; (2) A100—312 TFLOPS (624 with sparsity); (3) H100—989 TFLOPS (1,979 with sparsity); (4) B200—~2,250 TFLOPS; (5) GB200 (Grace Blackwell)—combined CPU+GPU system. Precision matters: (1) FP32—baseline FLOPS; (2) FP16/BF16—2× FP32 FLOPS (Tensor Cores); (3) FP8—2× FP16 FLOPS; (4) INT8—2-4× FP16 FLOPS; (5) INT4—2× INT8 FLOPS. FLOPS enables hardware comparison but real performance depends on memory bandwidth, interconnect, and software efficiency. LLM training compute: (1) FLOPs per token ≈ 6 × N (parameters) for forward + backward pass; (2) GPT-3 training: ~3.14 × 10²³ FLOPs; (3) LLaMA-70B: ~2.1 × 10²⁴ FLOPs (more data, Chinchilla-optimal). Model FLOPs utilization (MFU): ratio of achieved FLOPS to hardware peak—50-60% is good for LLM training (memory, communication overhead). Inference FLOPS: per-token generation requires ~2N FLOPs (forward pass only), but decode is usually memory-bound not compute-bound. Hardware comparison beyond FLOPS: memory bandwidth (bytes/s), memory capacity (GB), interconnect bandwidth (NVLink, InfiniBand), and TCO (total cost of ownership) equally important for AI workload selection. FLOPS provides the foundation for AI compute planning, cost estimation, and hardware selection decisions.