Processing-in-Memory (PIM) and Near-Data Processing is the computer architecture paradigm that moves computation to where the data resides rather than moving data to where the processor is — addressing the memory bandwidth wall by embedding compute units directly in or near memory (DRAM, HBM, storage), where data-intensive operations like search, aggregation, and simple arithmetic can execute at internal memory bandwidth (10-100× higher than external bus bandwidth) without the energy cost of data movement, which represents 60-90% of total energy in conventional architectures.

The Data Movement Problem

<svg viewBox="0 0 401 207" xmlns="http://www.w3.org/2000/svg" style="max-width:100%;height:auto" role="img"><rect x="0" y="0" width="401" height="207" rx="12" fill="#0d1117"/><g font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,&quot;Liberation Mono&quot;,monospace" font-size="14"><text xml:space="preserve" x="20" y="31.7"><tspan fill="#c9d1d9"> Conventional:</tspan></text><text xml:space="preserve" x="20" y="50.7"><tspan fill="#c9d1d9"> [CPU/GPU] </tspan><tspan fill="#6e7681">←──</tspan><tspan fill="#c9d1d9"> external bus </tspan><tspan fill="#6e7681">──→</tspan><tspan fill="#c9d1d9"> [DRAM]</tspan></text><text xml:space="preserve" x="20" y="69.7"><tspan fill="#c9d1d9">              64-128 GB/s</tspan></text><text xml:space="preserve" x="20" y="88.7"><tspan fill="#c9d1d9">              ~10 pJ/bit transfer energy</tspan></text><text xml:space="preserve" x="20" y="107.7"></text><text xml:space="preserve" x="20" y="126.7"><tspan fill="#c9d1d9"> Processing-in-Memory:</tspan></text><text xml:space="preserve" x="20" y="145.7"><tspan fill="#c9d1d9"> [DRAM + embedded compute]</tspan></text><text xml:space="preserve" x="20" y="164.7"><tspan fill="#c9d1d9">              Internal bandwidth: 1-10 TB/s</tspan></text><text xml:space="preserve" x="20" y="183.7"><tspan fill="#c9d1d9">              ~0.1 pJ/bit (no bus transfer)</tspan></text></g></svg>

• Modern CPUs: 50% of power spent on data movement (not computation).
• GPU HBM: 3.35 TB/s bandwidth (H100) → still not enough for many workloads.
• PIM: Use the massive internal bandwidth of DRAM banks (each bank: ~10-50 GB/s, 32 banks = 320-1600 GB/s).

PIM Approaches

Samsung HBM-PIM

<svg viewBox="0 0 536 245" xmlns="http://www.w3.org/2000/svg" style="max-width:100%;height:auto" role="img"><rect x="0" y="0" width="536" height="245" rx="12" fill="#0d1117"/><g font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,&quot;Liberation Mono&quot;,monospace" font-size="14"><text xml:space="preserve" x="20" y="31.7"><tspan fill="#c9d1d9"> HBM Stack:</tspan></text><text xml:space="preserve" x="20" y="50.7"><tspan fill="#c9d1d9"> </tspan><tspan fill="#6e7681">┌─────────────────┐</tspan></text><text xml:space="preserve" x="20" y="69.7"><tspan fill="#c9d1d9"> </tspan><tspan fill="#6e7681">│</tspan><tspan fill="#c9d1d9">   DRAM Die 3    </tspan><tspan fill="#6e7681">│</tspan></text><text xml:space="preserve" x="20" y="88.7"><tspan fill="#c9d1d9"> </tspan><tspan fill="#6e7681">│</tspan><tspan fill="#c9d1d9">   DRAM Die 2    </tspan><tspan fill="#6e7681">│</tspan><tspan fill="#c9d1d9">  Each die bank has small FP16 ALU</tspan></text><text xml:space="preserve" x="20" y="107.7"><tspan fill="#c9d1d9"> </tspan><tspan fill="#6e7681">│</tspan><tspan fill="#c9d1d9">   DRAM Die 1    </tspan><tspan fill="#6e7681">│</tspan><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">→</tspan><tspan fill="#c9d1d9"> Process data without sending to GPU</tspan></text><text xml:space="preserve" x="20" y="126.7"><tspan fill="#c9d1d9"> </tspan><tspan fill="#6e7681">│</tspan><tspan fill="#c9d1d9">   DRAM Die 0    </tspan><tspan fill="#6e7681">│</tspan></text><text xml:space="preserve" x="20" y="145.7"><tspan fill="#c9d1d9"> </tspan><tspan fill="#6e7681">│</tspan><tspan fill="#c9d1d9">  Base Logic Die </tspan><tspan fill="#6e7681">│</tspan><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">←</tspan><tspan fill="#c9d1d9"> PIM controller + ALUs</tspan></text><text xml:space="preserve" x="20" y="164.7"><tspan fill="#c9d1d9"> </tspan><tspan fill="#6e7681">└─────────────────┘</tspan></text><text xml:space="preserve" x="20" y="183.7"></text><text xml:space="preserve" x="20" y="202.7"><tspan fill="#c9d1d9"> Optimized for: Element-wise ops, GEMV, embedding lookups</tspan></text><text xml:space="preserve" x="20" y="221.7"><tspan fill="#c9d1d9"> Bandwidth: ~1 TB/s internal (vs. 3.35 TB/s external HBM3)</tspan></text></g></svg>

UPMEM: Commercial PIM

• DIMM-compatible PIM: Replace standard DDR DIMMs with PIM DIMMs.
• Each DIMM: 2,560 processing elements (DPUs), each with:
• 32-bit RISC core, 24 KB instruction mem, 64 KB working mem.
• Direct access to 64 MB MRAM.
• Applications: Genomics (sequence matching), databases (scan/filter), analytics.

PIM-Suitable Workloads

PIM-Unsuitable Workloads

Energy Efficiency

Processing-in-memory is the architectural response to the data movement crisis that dominates modern computing energy budgets — by embedding computation within the memory hierarchy itself, PIM eliminates the fundamental bottleneck of moving data across bandwidth-limited buses, offering order-of-magnitude improvements in energy efficiency and throughput for data-intensive workloads, and representing a potential paradigm shift as memory bandwidth demands continue to outpace interconnect scaling.