compute bound
**Compute bound describes a workload whose elapsed time is limited primarily by available arithmetic execution rather than data movement or I/O.** It tells engineers that more bandwidth alone will not improve performance and that arithmetic units, instruction mix, precision, and utilization deserve attention. In the roofline model, a kernel is compute bound when its arithmetic intensity exceeds machine balance, the ratio of peak operations per second to peak bytes per second. A professional performance claim defines workload, useful work, input and output shapes, numerical format, batch and concurrency, warmup and measurement interval, hardware and software versions, power state, correctness tolerance, and aggregation method. Peak specifications are ceilings under particular conditions; delivered behavior includes utilization, data movement, synchronization, control overhead, and tail effects. The classification belongs to a specific workload shape, implementation, precision, cache behavior, device, and concurrency; the same operation can cross the ridge point when batch or tiling changes.
**Architecture, quantitative model, and operating behavior.** Attainable performance is bounded by the smaller of peak compute and arithmetic intensity times bandwidth. Above the ridge, the horizontal compute roof dominates. Additional ceilings from instruction issue, occupancy, dependencies, special functions, or tensor-core eligibility may sit below nominal peak. Large dense GEMMs and convolutions with enough batch and reuse are common examples. They keep operands local across many multiply-accumulates and issue enough independent tiles to occupy arithmetic pipelines. Tensor-core bound, vector-ALU bound, scalar/control bound, instruction-throughput bound, dependency-latency bound, and mixed compute/communication regimes need different remedies even though counters show high compute activity. Useful analysis separates arithmetic, memory hierarchy, interconnect, storage, control, and queuing. It counts operations and bytes at each boundary, identifies dependencies and reuse, estimates ideal ceilings, and then uses counters and traces to explain the gap between the model and measurement. Ratios without a clearly named numerator and denominator invite invalid comparisons. Report useful throughput together with latency distribution, utilization, arithmetic intensity, achieved bandwidth, cache hit rate, occupancy, communication time, memory capacity, power, energy per result, quality, and cost. Include median and tail behavior, sustained rather than burst operation, repeated trials, and uncertainty. A faster approximation is not equivalent unless it meets the same accuracy and service constraints.
**Implementation, hardware mapping, and bottlenecks.** Use faster algorithms, supported lower precision, fused multiply-accumulate tiles, vectorization, instruction-level parallelism, occupancy, load balancing, and kernel fusion only where fusion does not increase register pressure enough to lower execution rate. More tensor cores or TOPS help only when kernels map to them and sustained clocks, power, register files, and operand collectors keep them fed. FP8 may raise peak compute, shifting the ridge and making a formerly compute-bound kernel bandwidth-bound. High utilization alone can reflect inefficient extra work; low bandwidth does not prove compute limitation; quoting peak sparse rates for dense work or changing numeric quality invalidates conclusions. Begin with a correct reference and representative shapes. Profile end to end, classify the dominant resource, inspect kernel and system timelines, change one bottleneck at a time, and remeasure because optimization moves pressure elsewhere. Tiling, fusion, batching, vectorization, layout, precision, compression, overlap, prefetch, sharding, and algorithm choice are useful only when they reduce the limiting resource. The execution path spans registers, local SRAM and caches, HBM or GDDR, host DRAM, PCIe or coherent links, scale-up fabric, network, and storage. Compute units consume tensors only when compilers and kernels issue enough independent work and the hierarchy supplies operands. Package wiring, memory stacks, clocks, voltage, thermal headroom, and power delivery determine sustained limits. Frequent mistakes include quoting peak instead of achieved rates, omitting data conversion and transfer, measuring a cached toy input, timing asynchronous work without synchronization, mixing decimal and binary units, ignoring warmup or throttling, changing precision or quality, averaging away tails, and optimizing a component that is not on the critical path.
**Measurement, validation, and engineering controls.** Measure operations, bytes, instruction mix, tensor-core active cycles, issue stalls, occupancy, achieved FLOPS, and scaling with clock or compute units; vary bandwidth and intensity to confirm sensitivity. Arithmetic intensity, achieved/peak FLOPS, tensor utilization, instruction throughput, occupancy, clock, power, latency, throughput, and numerical quality matter. Place the measured point on a calibrated roofline and inspect lower ceilings; sweep tile and batch sizes to see whether performance approaches the flat roof. Verification combines analytical bounds, microbenchmarks, hardware counters, kernel timelines, end-to-end traces, scaling sweeps, sensitivity to batch and shape, cold and warm runs, long-duration thermal tests, correctness comparisons, fault and congestion tests, and independent reproduction. Roofline and queueing models guide diagnosis but must be calibrated against the deployed machine. Benchmark code, datasets, model and compiler artifacts, drivers, firmware, topology, clock and power settings, environment, commands, raw samples, counter traces, and analysis notebooks remain versioned. Continuous tests detect regressions in quality, latency, throughput, bandwidth, memory, power, and cost, with thresholds chosen from variance rather than a single run. Published comparisons disclose configuration, exclusions, tuning effort, measurement boundary, quality criteria, and uncertainty. Energy and carbon claims distinguish chip, IT, and facility boundaries and avoid extrapolating one benchmark to all workloads. Owners review regressions and retain evidence sufficient to reproduce decisions.
| Workload/operation | Typical intensity | Likely limit | Primary lever | Verification |
|---|---|---|---|---|
| Large batched GEMM | High | Compute throughput | Tensor cores/precision/tiles | Roofline and instruction rate |
| LLM single-token decode | Low | Weight-memory bandwidth | Batch/compress/cache | Bytes per token |
| Elementwise chain | Low unless fused | Memory traffic/launch | Fusion/vectorization | Kernel and byte count |
| Large convolution | Moderate to high | Shape dependent | Tiling/algorithm | Sweep batch/channels |
| Embedding lookup | Very low/irregular | Memory latency/bandwidth | Layout/cache/shard | Hit rate and stalls |
| Collective communication | Network intensity | Fabric/latency | Topology/overlap/compress | Link counters/timeline |
```svg
```
**Selection and system-level application.** Optimize compute only after measured evidence places the workload beyond the ridge; otherwise increase reuse or bandwidth first. Large-batch training GEMMs, dense convolution, scientific dense linear algebra, and compute-heavy simulation are often compute bound. Compute balance depends on precision, batching, sparsity, fusion, memory, compiler, power and cooling, and distributed communication. Optimization is a system exercise across algorithms, precision, kernels, compiler, runtime, accelerator, memory, interconnect, scheduler, serving policy, cooling, and facility limits. Removing one ceiling often exposes another, so architecture decisions should optimize time and energy to a useful result rather than an isolated metric. A professional performance claim defines workload, useful work, input and output shapes, numerical format, batch and concurrency, warmup and measurement interval, hardware and software versions, power state, correctness tolerance, and aggregation method. Peak specifications are ceilings under particular conditions; delivered behavior includes utilization, data movement, synchronization, control overhead, and tail effects. Report useful throughput together with latency distribution, utilization, arithmetic intensity, achieved bandwidth, cache hit rate, occupancy, communication time, memory capacity, power, energy per result, quality, and cost. Include median and tail behavior, sustained rather than burst operation, repeated trials, and uncertainty. A faster approximation is not equivalent unless it meets the same accuracy and service constraints. CFS connects this topic to semiconductor architecture, implementation, verification, manufacturing, packaging, test, and deployed AI-system tradeoffs across the platform.