in-memory computing
**In-memory computing performs selected operations inside or immediately beside memory arrays, reducing the movement of operands between storage and arithmetic units.** This attacks the memory wall: reading weights can cost far more energy and time than multiplying them. Analog crossbars exploit physical current summation for matrix-vector multiplication, while digital processing-in-memory (PIM) adds logic near SRAM, DRAM, or HBM. The goal is locality, not the elimination of processors; control flow, nonlinear functions, and exact arithmetic often remain conventional.
**Analog crossbars apply Ohm’s and Kirchhoff’s laws as computation.** Conductance \(G_{ij}\) stores a weight at row \(i\), column \(j\). Input voltages \(V_i\) drive word lines, and bit-line current approximates \(I_j=\sum_i V_iG_{ij}\). Many multiply-accumulates occur simultaneously in one physical step. ReRAM, phase-change memory (PCM), flash, and analog SRAM cells offer different retention, programmability, variability, and integration options.
| PIM approach | Compute location and representation | Strength | Limitation |
|---|---|---|---|
| Analog ReRAM/PCM | Conductance crossbar, analog current | Exceptional parallel MAC density | Variability, endurance, ADC/DAC overhead |
| SRAM compute-in-memory | Modified SRAM bit cells and periphery | CMOS maturity and fast access | Area overhead and often modest precision |
| HBM-PIM | Digital engines inside stacked-memory logic/DRAM vicinity | High internal bandwidth for existing data | Programming model and limited operation set |
| Near-memory accelerator | Logic die or chiplet beside HBM/DRAM | Flexible digital precision | Still moves data across local interfaces |
| UPMEM-style DRAM PIM | Many small processors near DRAM banks | Good for data-parallel scans and transforms | Per-core performance and explicit programming |
**A crossbar is not a complete accelerator.** Row drivers and DACs encode inputs, selectors prevent sneak paths, sense amplifiers and ADCs digitize columns, shift-add logic combines bit slices, and buffers stage activations. Digital units apply activation, normalization, pooling, and accumulation. Peripheral circuitry can occupy more area and energy than memory cells, so array size and converter sharing are core architectural decisions.
```svg
```
**Weights commonly use differential and bit-sliced encoding.** Two cells represent positive and negative components, and several low-bit arrays combine into a higher-precision value. Inputs may also be processed one bit plane at a time. These choices multiply cycles and array area relative to an ideal one-cell MAC, but improve linearity and handle signed values. Mapping software accounts for stuck cells, conductance ranges, and ADC saturation.
**Device nonidealities shape algorithm accuracy.** Program noise, read noise, drift, finite conductance levels, nonlinear I–V behavior, line resistance, sneak current, and temperature distort multiplication. PCM conductance drifts with time; ReRAM switching varies; SRAM designs face transistor mismatch. Write-verify algorithms, periodic calibration, redundancy, hardware-aware training, and error compensation keep models within accuracy targets.
**Endurance separates inference from frequent learning.** Inference weights change infrequently and suit nonvolatile arrays with limited write cycles. Training updates weights constantly and demands symmetric, linear, fine-grained changes that physical devices rarely provide. Hybrid training accumulates gradients digitally and writes less often, or uses analog devices for forward operations while maintaining a high-precision shadow copy.
**Digital PIM preserves exactness and familiar logic.** Small ALUs in memory banks can perform vector arithmetic, search, filtering, bit operations, or reductions without exporting every cache line. Samsung demonstrated HBM-PIM concepts, while UPMEM places programmable cores near DRAM banks. These systems sacrifice some analog density but avoid converter error and support irregular data processing beyond matrix multiplication.
**SRAM compute-in-memory integrates naturally into SoCs.** Modified bit cells or sense amplifiers compute XNOR, AND, accumulation, or multi-bit dot products along bit lines. SRAM is fast and durable, but it is less dense and volatile. Designers balance read stability, compute margin, memory capacity, and conventional access. Edge inference benefits because weights and compute remain on chip, eliminating external DRAM energy.
**AI inference is attractive because weights dominate traffic.** A model layer reuses an activation vector across many stationary weights in the array. Large language models remain challenging because capacity, KV-cache movement, attention, and flexible operators exceed a single array. PIM may accelerate projections, embeddings, search, or bandwidth-heavy kernels while GPUs handle the rest. Sparsity can help only if the architecture avoids activating unused rows and columns.
**Data layout and scheduling determine utilization.** Matrices are partitioned across finite arrays; padding wastes cells; convolution lowering can duplicate data. Batch size, tile shape, precision, and ADC sharing affect throughput. Compilers place weights, insert scaling, schedule conversions, and move intermediate results. Remapping around defects and wear must remain transparent to the model.
**Memory consistency matters for general-purpose PIM.** CPU caches may hold stale copies while a near-memory engine modifies DRAM. Systems flush ranges, use coherent protocols, allocate uncached buffers, or expose explicit ownership. Virtual memory, protection, interrupts, and context switching complicate programmable PIM. A narrow accelerator interface is easier to deploy but less flexible.
**Benchmark claims must include peripherals and accuracy.** Raw crossbar operations can imply enormous TOPS/W, yet DACs, ADCs, buffers, inter-array movement, control, and writes reduce delivered efficiency. Comparisons should state technology, precision, sparsity, array utilization, batch, model accuracy, and whether off-chip traffic is counted. End-to-end latency and energy on a real network are the meaningful results.
**Manufacturing and test need new observability.** Arrays contain distributions rather than identical digital bits. Wafer test measures conductance window, write yield, endurance proxies, retention, line resistance, and converter offsets. Built-in calibration records per-tile parameters. Error maps support repair and placement. Integrating emerging memory above CMOS must respect thermal budgets and contamination rules.
**In-memory computing is best understood as a hierarchy choice.** It moves arithmetic to the level where data already lives, from SRAM bit cells to HBM logic dies. Analog implementations maximize physical parallelism; digital PIM maximizes robustness and programmability. Successful products use locality where it outweighs conversion, precision, capacity, endurance, and software costs, usually as part of a heterogeneous system.
**Power delivery and thermal density constrain simultaneous array use.** A crossbar operation activates many rows and converters together, creating local current demand even if energy per MAC is low. IR drop can alter applied voltage and therefore the computed value. Controllers stagger tiles, regulate reference voltages, monitor temperature, and incorporate supply variation into calibration. Dense three-dimensional memory worsens heat removal, so peak theoretical parallelism may exceed sustainable throughput.
**Security changes when data and execution share a physical array.** Access control must prevent one tenant from invoking compute on another tenant’s rows. Residual analog state, remanence, fault injection, and side channels through current or timing require threat analysis. Secure erase, encrypted external traffic, attested firmware, and partitioned calibration metadata protect deployment. Approximate arithmetic does not relax isolation or data-integrity requirements.
**Adoption requires an API that exposes benefit without exposing device quirks.** Libraries can present matrix operations, scans, reductions, or database primitives while compilers choose conventional or PIM execution. Profilers must report movement saved, conversion cost, array occupancy, endurance consumption, and accuracy loss. Stable abstractions let device technology evolve beneath applications and allow safe fallback when a layer cannot meet precision.