gustafson law
**Gustafson's law models scaled speedup when the parallel portion of a problem grows with available processors while serial time remains comparatively fixed.** It explains why larger clusters can productively train larger models, process more data, or simulate finer grids even when a fixed problem would hit Amdahl limits. For N processors and serial fraction alpha measured on the parallel run, scaled speedup is N minus alpha times N minus one, approaching linear growth when alpha is small. A professional performance claim defines workload, useful work, input and output shapes, numerical format, batch and concurrency, warmup and measurement interval, hardware and software versions, power state, correctness tolerance, and aggregation method. Peak specifications are ceilings under particular conditions; delivered behavior includes utilization, data movement, synchronization, control overhead, and tail effects. This is a weak-scaling perspective: problem size and useful parallel work increase with resources, while time-to-solution may remain near a target. It does not eliminate communication, memory, data, convergence, or cost limits.
**Architecture, quantitative model, and operating behavior.** Amdahl asks how fast the same job completes; Gustafson asks how much larger a job completes in similar time. AI uses this idea when more GPUs support larger global batches, models, sequence lengths, experts, datasets, or experiment throughput. Choose a scaled workload rule, hold relevant work per processor or elapsed time target, measure serial and parallel portions, grow N and problem size, then evaluate scaled speedup, weak-scaling efficiency, quality, and total cost. Weak scaling can keep examples, parameters, spatial cells, tokens, or memory per processor constant. Statistical scaling in ML is more complex because optimization steps and sample efficiency may change with global batch or model size. Useful analysis separates arithmetic, memory hierarchy, interconnect, storage, control, and queuing. It counts operations and bytes at each boundary, identifies dependencies and reuse, estimates ideal ceilings, and then uses counters and traces to explain the gap between the model and measurement. Ratios without a clearly named numerator and denominator invite invalid comparisons. Report useful throughput together with latency distribution, utilization, arithmetic intensity, achieved bandwidth, cache hit rate, occupancy, communication time, memory capacity, power, energy per result, quality, and cost. Include median and tail behavior, sustained rather than burst operation, repeated trials, and uncertainty. A faster approximation is not equivalent unless it meets the same accuracy and service constraints.
**Implementation, hardware mapping, and bottlenecks.** Partition data and model state, keep communication proportional, map parallel dimensions to topology, overlap collectives, scale input and checkpoints, preserve optimizer behavior, and ensure larger problems provide genuine utility. Additional accelerators bring HBM and compute but also demand fabric bisection, host capacity, storage, power, cooling, and reliable scheduling. Expert all-to-all or global reductions can destroy weak scaling. Calling a bigger benchmark a speedup without equal utility, ignoring convergence changes, assuming serial fraction is constant, hiding added energy and cost, or exceeding data/model scaling benefits overstates success. Begin with a correct reference and representative shapes. Profile end to end, classify the dominant resource, inspect kernel and system timelines, change one bottleneck at a time, and remeasure because optimization moves pressure elsewhere. Tiling, fusion, batching, vectorization, layout, precision, compression, overlap, prefetch, sharding, and algorithm choice are useful only when they reduce the limiting resource. The execution path spans registers, local SRAM and caches, HBM or GDDR, host DRAM, PCIe or coherent links, scale-up fabric, network, and storage. Compute units consume tensors only when compilers and kernels issue enough independent work and the hierarchy supplies operands. Package wiring, memory stacks, clocks, voltage, thermal headroom, and power delivery determine sustained limits. Frequent mistakes include quoting peak instead of achieved rates, omitting data conversion and transfer, measuring a cached toy input, timing asynchronous work without synchronization, mixing decimal and binary units, ignoring warmup or throttling, changing precision or quality, averaging away tails, and optimizing a component that is not on the critical path.
**Measurement, validation, and engineering controls.** Define the scaled problem, measure fixed elapsed-time utility, weak-scaling efficiency, communication and serial time, quality to target, energy and cost; compare with a fixed-size Amdahl run. Scaled speedup, work per processor, weak-scaling efficiency, time to quality, model/data scale, communication ratio, utilization, energy, cost, and fault rate matter. Phase timelines across N reveal whether input, collective, checkpoint, evaluation, or scheduling grows faster than useful work. Verification combines analytical bounds, microbenchmarks, hardware counters, kernel timelines, end-to-end traces, scaling sweeps, sensitivity to batch and shape, cold and warm runs, long-duration thermal tests, correctness comparisons, fault and congestion tests, and independent reproduction. Roofline and queueing models guide diagnosis but must be calibrated against the deployed machine. Benchmark code, datasets, model and compiler artifacts, drivers, firmware, topology, clock and power settings, environment, commands, raw samples, counter traces, and analysis notebooks remain versioned. Continuous tests detect regressions in quality, latency, throughput, bandwidth, memory, power, and cost, with thresholds chosen from variance rather than a single run. Published comparisons disclose configuration, exclusions, tuning effort, measurement boundary, quality criteria, and uncertainty. Energy and carbon claims distinguish chip, IT, and facility boundaries and avoid extrapolating one benchmark to all workloads. Owners review regressions and retain evidence sufficient to reproduce decisions.
| Dimension | Amdahl perspective | Gustafson perspective | AI example | Decision metric |
|---|---|---|---|---|
| Problem size | Fixed | Scales with N | Same model versus larger model/data | Latency versus capability |
| Core formula | 1 divided by serial plus parallel/N | N minus serial fraction times N minus one | Strong versus weak scale | Speedup definition |
| Asymptotic behavior | Plateaus at serial limit | Near-linear if serial share stays small | More GPUs for more useful work | Efficiency |
| Primary risk | Serial bottleneck | Scaling overhead/data utility | Collectives or batch degradation | Time to quality |
| Best planning use | Fixed-job acceleration | Capacity/problem expansion | Serving latency versus pretraining scale | Cost and energy |
```svg
```
**Selection and system-level application.** Use Gustafson for capacity planning and science or AI workloads where a larger problem is valuable; use Amdahl for fixed-latency objectives, and report both when decisions span capacity and response time. Foundation-model training, hyperparameter sweeps, high-resolution simulation, rendering, graph processing, data analytics, and ensemble inference use scaled-problem reasoning. Weak scaling depends on algorithms, model/data laws, parallel decomposition, network, memory, storage, scheduler, power, cooling, reliability, and economics. Optimization is a system exercise across algorithms, precision, kernels, compiler, runtime, accelerator, memory, interconnect, scheduler, serving policy, cooling, and facility limits. Removing one ceiling often exposes another, so architecture decisions should optimize time and energy to a useful result rather than an isolated metric. A professional performance claim defines workload, useful work, input and output shapes, numerical format, batch and concurrency, warmup and measurement interval, hardware and software versions, power state, correctness tolerance, and aggregation method. Peak specifications are ceilings under particular conditions; delivered behavior includes utilization, data movement, synchronization, control overhead, and tail effects. Report useful throughput together with latency distribution, utilization, arithmetic intensity, achieved bandwidth, cache hit rate, occupancy, communication time, memory capacity, power, energy per result, quality, and cost. Include median and tail behavior, sustained rather than burst operation, repeated trials, and uncertainty. A faster approximation is not equivalent unless it meets the same accuracy and service constraints. CFS connects this topic to semiconductor architecture, implementation, verification, manufacturing, packaging, test, and deployed AI-system tradeoffs across the platform.