Memory bound describes a workload whose elapsed time is limited primarily by data supply rather than arithmetic capability. It explains why enormous advertised FLOPS can remain idle during LLM decode, embeddings, elementwise transforms, reductions, sparse operations, and irregular analytics. In the roofline model, arithmetic intensity below machine balance places a kernel on the rising bandwidth roof, where attainable operations per second equal intensity times sustained bytes per second. A professional performance claim defines workload, useful work, input and output shapes, numerical format, batch and concurrency, warmup and measurement interval, hardware and software versions, power state, correctness tolerance, and aggregation method. Peak specifications are ceilings under particular conditions; delivered behavior includes utilization, data movement, synchronization, control overhead, and tail effects. The boundary must name the memory level: a kernel can be HBM bound, cache bound, shared-memory bank bound, host-memory bound, or interconnect bound.

Architecture, quantitative model, and operating behavior. Low-reuse operations fetch many bytes for little arithmetic. Single-token autoregressive decode reads large weight matrices for a small amount of output; batching reuses weights across more tokens and raises intensity. Fusion avoids intermediate writes and moves the point rightward. Caches, prefetch, coalescing, tiling, compression, quantization, sparsity, batching, and recomputation trade capacity, compute, or latency for fewer external bytes. Higher-bandwidth HBM raises the diagonal roof but does not correct random latency or poor locality by itself. Bandwidth-bound streaming, latency-bound pointer chasing, capacity-bound paging, cache-thrashing, TLB-bound, and network-bound regimes may all starve compute but require different evidence and solutions. Useful analysis separates arithmetic, memory hierarchy, interconnect, storage, control, and queuing. It counts operations and bytes at each boundary, identifies dependencies and reuse, estimates ideal ceilings, and then uses counters and traces to explain the gap between the model and measurement. Ratios without a clearly named numerator and denominator invite invalid comparisons. Report useful throughput together with latency distribution, utilization, arithmetic intensity, achieved bandwidth, cache hit rate, occupancy, communication time, memory capacity, power, energy per result, quality, and cost. Include median and tail behavior, sustained rather than burst operation, repeated trials, and uncertainty. A faster approximation is not equivalent unless it meets the same accuracy and service constraints.

Implementation, hardware mapping, and bottlenecks. Count mandatory and avoidable bytes, fuse elementwise chains, retain tiles locally, quantize weights and KV cache, batch compatible requests, reorganize layouts, prefetch predictably, and shard so traffic follows topology. HBM stacks and memory controllers help streaming demand; larger caches help reuse; more channels help parallelism; on-chip SRAM enables explicit tiling. Narrow links or package power can become the true memory ceiling. Calling every low-utilization kernel memory bound, optimizing FLOPS, compressing data with excessive decode cost, batching beyond latency SLO, or ignoring write amplification and cache misses leads to wrong action. Begin with a correct reference and representative shapes. Profile end to end, classify the dominant resource, inspect kernel and system timelines, change one bottleneck at a time, and remeasure because optimization moves pressure elsewhere. Tiling, fusion, batching, vectorization, layout, precision, compression, overlap, prefetch, sharding, and algorithm choice are useful only when they reduce the limiting resource. The execution path spans registers, local SRAM and caches, HBM or GDDR, host DRAM, PCIe or coherent links, scale-up fabric, network, and storage. Compute units consume tensors only when compilers and kernels issue enough independent work and the hierarchy supplies operands. Package wiring, memory stacks, clocks, voltage, thermal headroom, and power delivery determine sustained limits. Frequent mistakes include quoting peak instead of achieved rates, omitting data conversion and transfer, measuring a cached toy input, timing asynchronous work without synchronization, mixing decimal and binary units, ignoring warmup or throttling, changing precision or quality, averaging away tails, and optimizing a component that is not on the critical path.

Measurement, validation, and engineering controls. Measure bytes at cache/HBM/interconnect levels, achieved bandwidth, stall reasons, latency, cache and TLB hit rates, and sensitivity to bandwidth, clocks, batch, compression, and reuse. Bytes per useful result, arithmetic intensity, achieved/peak bandwidth, compute idle fraction, cache/TLB behavior, batch, latency, throughput, energy, and quality matter. A byte accounting model reconciled with counters separates mandatory model reads from spills, layout copies, allocator traffic, and redundant intermediates. Verification combines analytical bounds, microbenchmarks, hardware counters, kernel timelines, end-to-end traces, scaling sweeps, sensitivity to batch and shape, cold and warm runs, long-duration thermal tests, correctness comparisons, fault and congestion tests, and independent reproduction. Roofline and queueing models guide diagnosis but must be calibrated against the deployed machine. Benchmark code, datasets, model and compiler artifacts, drivers, firmware, topology, clock and power settings, environment, commands, raw samples, counter traces, and analysis notebooks remain versioned. Continuous tests detect regressions in quality, latency, throughput, bandwidth, memory, power, and cost, with thresholds chosen from variance rather than a single run. Published comparisons disclose configuration, exclusions, tuning effort, measurement boundary, quality criteria, and uncertainty. Energy and carbon claims distinguish chip, IT, and facility boundaries and avoid extrapolating one benchmark to all workloads. Owners review regressions and retain evidence sufficient to reproduce decisions.

<svg viewBox="0 0 760 470" xmlns="http://www.w3.org/2000/svg" font-family="-apple-system,Segoe UI,Roboto,sans-serif">
  <rect width="760" height="470" fill="#0d1117"/>
  <defs>
    <marker id="mbBlueArrow" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto">
      <path d="M0 0L10 5L0 10Z" fill="#60a5fa"/>
    </marker>
    <marker id="mbGreenArrow" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto">
      <path d="M0 0L10 5L0 10Z" fill="#34d399"/>
    </marker>
    <marker id="mbGoldArrow" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto">
      <path d="M0 0L10 5L0 10Z" fill="#f59e0b"/>
    </marker>
    <filter id="mbKernelGlow" x="-80%" y="-80%" width="260%" height="260%">
      <feGaussianBlur stdDeviation="4" result="blur"/>
      <feMerge><feMergeNode in="blur"/><feMergeNode in="SourceGraphic"/></feMerge>
    </filter>
  </defs>

  <text x="380" y="31" fill="#e6edf3" font-size="21" font-weight="700" text-anchor="middle">Memory-Bound Performance — Compute Waits for Data</text>
  <text x="380" y="53" fill="#8b98a5" font-size="12.5" text-anchor="middle">when each byte does little work, bandwidth—not arithmetic throughput—sets the performance ceiling</text>

  <!-- Physical bottleneck: wide compute, narrow off-chip path -->
  <g transform="translate(24 87)">
    <text x="224" y="-9" fill="#93c5fd" font-size="10.5" font-weight="700" text-anchor="middle">WHY EXECUTION UNITS GO IDLE</text>
    <rect width="448" height="251" rx="13" fill="#0a1421" stroke="#60a5fa" stroke-width="1.6"/>

    <!-- DRAM supplies cache lines across constrained bandwidth -->
    <g transform="translate(14 30)">
      <text x="47" y="-9" fill="#fbbf24" font-size="9.5" font-weight="700" text-anchor="middle">OFF-CHIP DRAM</text>
      <g fill="#32270f" stroke="#f59e0b">
        <rect x="0" y="0" width="94" height="31" rx="4"/><rect x="4" y="35" width="94" height="31" rx="4"/><rect x="8" y="70" width="94" height="31" rx="4"/><rect x="12" y="105" width="94" height="31" rx="4"/>
      </g>
      <g fill="#fbbf24" font-size="7.5" text-anchor="middle">
        <text x="47" y="19">memory bank 0</text><text x="51" y="54">memory bank 1</text><text x="55" y="89">memory bank 2</text><text x="59" y="124">memory bank 3</text>
      </g>
      <text x="54" y="154" fill="#8b98a5" font-size="8" text-anchor="middle">~100 ns latency</text>
    </g>

    <!-- Narrow memory link and request queue -->
    <path d="M116 102H175" stroke="#f59e0b" stroke-width="5" marker-end="url(#mbGoldArrow)"/>
    <path d="M175 118H116" stroke="#60a5fa" stroke-width="5" marker-end="url(#mbBlueArrow)"/>
    <text x="145" y="91" fill="#fbbf24" font-size="8.2" text-anchor="middle">requests</text>
    <text x="145" y="137" fill="#93c5fd" font-size="8.2" text-anchor="middle">cache lines</text>
    <g transform="translate(126 152)">
      <rect width="40" height="53" rx="5" fill="#211318" stroke="#f87171"/>
      <g fill="#32191e" stroke="#f87171"><rect x="6" y="7" width="28" height="8"/><rect x="6" y="19" width="28" height="8"/><rect x="6" y="31" width="28" height="8"/></g>
      <text x="20" y="49" fill="#fca5a5" font-size="7" text-anchor="middle">queue</text>
    </g>

    <!-- On-chip tile/cache can be reused -->
    <g transform="translate(181 35)">
      <text x="64" y="-9" fill="#6ee7b7" font-size="9.5" font-weight="700" text-anchor="middle">ON-CHIP TILE / CACHE</text>
      <rect width="128" height="115" rx="9" fill="#10251f" stroke="#34d399"/>
      <g transform="translate(14 14)" fill="#15382d" stroke="#34d399">
        <rect width="21" height="21"/><rect x="24" width="21" height="21"/><rect x="48" width="21" height="21"/><rect x="72" width="21" height="21"/>
        <rect y="24" width="21" height="21"/><rect x="24" y="24" width="21" height="21"/><rect x="48" y="24" width="21" height="21"/><rect x="72" y="24" width="21" height="21"/>
        <rect y="48" width="21" height="21"/><rect x="24" y="48" width="21" height="21"/><rect x="48" y="48" width="21" height="21"/><rect x="72" y="48" width="21" height="21"/>
      </g>
      <path d="M24 92C50 111 91 106 105 83" fill="none" stroke="#34d399" stroke-width="2.5" marker-end="url(#mbGreenArrow)"/>
      <text x="64" y="108" fill="#6ee7b7" font-size="8" text-anchor="middle">reuse before eviction</text>
    </g>

    <!-- Wide compute array -->
    <g transform="translate(325 27)">
      <text x="55" y="-1" fill="#c4b5fd" font-size="9.5" font-weight="700" text-anchor="middle">COMPUTE LANES</text>
      <g stroke-width="1.4">
        <g fill="#153127" stroke="#34d399"><rect y="13" width="49" height="27" rx="4"/><rect x="57" y="13" width="49" height="27" rx="4"/></g>
        <g fill="#211318" stroke="#f87171"><rect y="47" width="49" height="27" rx="4"/><rect x="57" y="47" width="49" height="27" rx="4"/><rect y="81" width="49" height="27" rx="4"/><rect x="57" y="81" width="49" height="27" rx="4"/><rect y="115" width="49" height="27" rx="4"/><rect x="57" y="115" width="49" height="27" rx="4"/></g>
      </g>
      <g font-size="7.5" text-anchor="middle">
        <text x="24.5" y="31" fill="#6ee7b7">FMA</text><text x="81.5" y="31" fill="#6ee7b7">FMA</text>
        <text x="24.5" y="65" fill="#fca5a5">WAIT</text><text x="81.5" y="65" fill="#fca5a5">WAIT</text>
        <text x="24.5" y="99" fill="#fca5a5">WAIT</text><text x="81.5" y="99" fill="#fca5a5">WAIT</text>
        <text x="24.5" y="133" fill="#fca5a5">WAIT</text><text x="81.5" y="133" fill="#fca5a5">WAIT</text>
      </g>
      <text x="55" y="162" fill="#fca5a5" font-size="8.2" text-anchor="middle">6 / 8 lanes starved</text>
    </g>

    <path d="M309 91H324" stroke="#34d399" stroke-width="3" marker-end="url(#mbGreenArrow)"/>

    <!-- Arithmetic intensity definition -->
    <g transform="translate(181 167)">
      <rect width="238" height="57" rx="9" fill="#151322" stroke="#a78bfa"/>
      <text x="119" y="17" fill="#c4b5fd" font-size="9" font-weight="700" text-anchor="middle">ARITHMETIC INTENSITY</text>
      <text x="119" y="36" fill="#e6edf3" font-size="14" font-weight="700" text-anchor="middle">AI = useful FLOPs / bytes from DRAM</text>
      <text x="119" y="50" fill="#8b98a5" font-size="7.8" text-anchor="middle">more reuse raises AI without increasing off-chip traffic</text>
    </g>
  </g>

  <!-- Roofline model -->
  <g transform="translate(499 88)">
    <text x="117" y="-9" fill="#fbbf24" font-size="10.5" font-weight="700" text-anchor="middle">ROOFLINE MODEL</text>
    <rect width="236" height="250" rx="13" fill="#17150f" stroke="#f59e0b" stroke-width="1.6"/>
    <path d="M34 214V25M34 214H216" stroke="#3a4453"/>
    <text x="14" y="62" fill="#8b98a5" font-size="8" transform="rotate(-90 14 62)">performance · TFLOP/s</text>
    <text x="209" y="232" fill="#8b98a5" font-size="8" text-anchor="end">arithmetic intensity · FLOP/byte</text>

    <!-- Bandwidth slope and compute ceiling -->
    <path d="M39 208L157 57H214" fill="none" stroke="#f59e0b" stroke-width="3"/>
    <text x="91" y="132" fill="#fbbf24" font-size="8.2" transform="rotate(-52 91 132)">memory BW = 1 TB/s</text>
    <text x="207" y="51" fill="#6ee7b7" font-size="8.2" text-anchor="end">compute peak = 20 TFLOP/s</text>
    <path d="M157 42v179" stroke="#52647b" stroke-dasharray="4 3"/>
    <text x="157" y="238" fill="#8b98a5" font-size="7.8" text-anchor="middle">ridge = 20 FLOP/B</text>

    <!-- Original low-intensity kernel -->
    <circle cx="67" cy="172" r="8" fill="#3a181d" stroke="#f87171" stroke-width="2" filter="url(#mbKernelGlow)"/>
    <text x="72" y="190" fill="#fca5a5" font-size="8.2">AI 0.5</text>
    <text x="72" y="201" fill="#fca5a5" font-size="7.5">≤ 0.5 TFLOP/s</text>

    <!-- Reuse shifts right, then performance rises to the roof -->
    <path d="M74 169C100 159 130 132 175 62" fill="none" stroke="#34d399" stroke-width="2.5" stroke-dasharray="6 3" marker-end="url(#mbGreenArrow)"/>
    <circle cx="176" cy="57" r="8" fill="#153127" stroke="#34d399" stroke-width="2" filter="url(#mbKernelGlow)"/>
    <text x="207" y="74" fill="#6ee7b7" font-size="8.2" text-anchor="end">AI 32</text>
    <text x="207" y="85" fill="#6ee7b7" font-size="7.5" text-anchor="end">≤ 20 TFLOP/s</text>
    <g transform="translate(60 101)">
      <rect width="138" height="25" rx="12" fill="#10251f" stroke="#34d399"/>
      <text x="69" y="16" fill="#6ee7b7" font-size="8.2" text-anchor="middle">tile + reuse when algorithm allows</text>
    </g>

    <!-- Bound regions -->
    <g font-size="8" font-weight="700">
      <text x="44" y="227" fill="#fbbf24">MEMORY BOUND</text>
      <text x="161" y="36" fill="#6ee7b7">COMPUTE BOUND</text>
    </g>
  </g>

  <!-- Optimization logic: traffic first -->
  <g transform="translate(24 366)">
    <text x="356" y="-9" fill="#e6edf3" font-size="10.5" font-weight="700" text-anchor="middle">DIAGNOSE THE BOTTLENECK BEFORE OPTIMIZING</text>
    <rect width="711" height="67" rx="11" fill="#111820" stroke="#3a4453"/>
    <g transform="translate(17 15)">
      <rect width="155" height="36" rx="7" fill="#211318" stroke="#f87171"/>
      <text x="77.5" y="14" fill="#fca5a5" font-size="8.5" font-weight="700" text-anchor="middle">LOW BANDWIDTH UTILIZATION</text>
      <text x="77.5" y="28" fill="#8b98a5" font-size="7.5" text-anchor="middle">fix access pattern / latency / occupancy</text>
    </g>
    <path d="M179 33H206" stroke="#60a5fa" stroke-width="2" marker-end="url(#mbBlueArrow)"/>
    <g transform="translate(213 15)">
      <rect width="160" height="36" rx="7" fill="#211b0d" stroke="#f59e0b"/>
      <text x="80" y="14" fill="#fbbf24" font-size="8.5" font-weight="700" text-anchor="middle">BANDWIDTH SATURATED</text>
      <text x="80" y="28" fill="#8b98a5" font-size="7.5" text-anchor="middle">reduce bytes · fuse · compress · tile</text>
    </g>
    <path d="M380 33H407" stroke="#60a5fa" stroke-width="2" marker-end="url(#mbBlueArrow)"/>
    <g transform="translate(414 15)">
      <rect width="141" height="36" rx="7" fill="#10251f" stroke="#34d399"/>
      <text x="70.5" y="14" fill="#6ee7b7" font-size="8.5" font-weight="700" text-anchor="middle">AI ABOVE RIDGE</text>
      <text x="70.5" y="28" fill="#8b98a5" font-size="7.5" text-anchor="middle">optimize arithmetic throughput</text>
    </g>
    <path d="M562 33H589" stroke="#60a5fa" stroke-width="2" marker-end="url(#mbBlueArrow)"/>
    <g transform="translate(596 15)">
      <rect width="98" height="36" rx="7" fill="#151322" stroke="#a78bfa"/>
      <text x="49" y="14" fill="#c4b5fd" font-size="8.5" font-weight="700" text-anchor="middle">MEASURE AGAIN</text>
      <text x="49" y="28" fill="#8b98a5" font-size="7.5" text-anchor="middle">bottleneck can move</text>
    </g>
  </g>

  <text x="380" y="458" fill="#6b7684" font-size="11" text-anchor="middle">A kernel is memory bound only when measured performance is limited by data movement at its achieved arithmetic intensity.</text>
</svg>

Selection and system-level application. First reduce bytes and improve reuse, then add bandwidth; accept batching or compression only within latency and quality constraints. LLM token decode, embedding lookup, graph traversal, reductions, vector operations, sparse kernels, and streaming transforms are frequently memory bound. Memory limitation is co-designed with model architecture, precision, cache, batching, allocator, package, interconnect, serving policy, and power. Optimization is a system exercise across algorithms, precision, kernels, compiler, runtime, accelerator, memory, interconnect, scheduler, serving policy, cooling, and facility limits. Removing one ceiling often exposes another, so architecture decisions should optimize time and energy to a useful result rather than an isolated metric. A professional performance claim defines workload, useful work, input and output shapes, numerical format, batch and concurrency, warmup and measurement interval, hardware and software versions, power state, correctness tolerance, and aggregation method. Peak specifications are ceilings under particular conditions; delivered behavior includes utilization, data movement, synchronization, control overhead, and tail effects. Report useful throughput together with latency distribution, utilization, arithmetic intensity, achieved bandwidth, cache hit rate, occupancy, communication time, memory capacity, power, energy per result, quality, and cost. Include median and tail behavior, sustained rather than burst operation, repeated trials, and uncertainty. A faster approximation is not equivalent unless it meets the same accuracy and service constraints. CFS connects this topic to semiconductor architecture, implementation, verification, manufacturing, packaging, test, and deployed AI-system tradeoffs across the platform.