Compute-In-Memory

Compute-in-memory (CIM), also called processing-in-memory (PIM), is a processor architecture that performs computation directly inside or right next to the memory that holds the data, instead of shuttling operands back and forth to a separate arithmetic unit. By doing the math where the weights already sit, it attacks the dominant cost of modern AI hardware — moving bytes — rather than the arithmetic itself, and is especially suited to the dense multiply-accumulate (MAC) operations at the heart of neural networks.\n\n**It breaks the von Neumann separation of memory and compute.** A conventional machine keeps memory and the ALU apart and streams data across a bus between them; for memory-bound workloads that data movement dominates energy and latency, and the compute unit spends much of its time waiting. Compute-in-memory collapses that split: the storage array itself becomes the compute engine, so a weight never has to travel to a distant multiplier. The bottleneck the design targets is the bus, not the FLOPs.\n\n**Analog crossbars compute a dot-product with physics.** In the most striking form, each memory cell stores a weight as its conductance in a grid (a crossbar of SRAM, ReRAM, PCM, or flash). Drive a row with an input voltage and Ohm's law makes each cell pass a current equal to voltage times conductance; Kirchhoff's law then sums all currents on a shared column. One column therefore reads out an entire vector dot-product — a full MAC over many weights — in a single analog step, with no weights fetched and no per-element multiply. A whole matrix-vector product happens in the array at once.\n\n| | Von Neumann | Compute-in-memory |\n|---|---|---|\n| Where math happens | separate ALU | inside the memory array |\n| Data movement | weights stream over bus | weights stay put |\n| MAC cost | fetch + multiply + writeback | one analog column read |\n| Dominant limit | memory bandwidth / bus energy | ADC/DAC, analog noise |\n| Best fit | general-purpose, exact | dense MAC, tolerant precision |\n\n```svg\n\n```\n\n**The costs are precision, conversion, and generality.** Analog computation is noisy: device variation, drift, and limited cell states cap effective precision, and every array needs digital-to-analog drivers on the inputs and analog-to-digital converters on the outputs, whose energy and area can eat much of the savings. Write endurance and non-linearity limit which memory technologies work, and the paradigm fits dense, low-precision MAC-heavy layers far better than exact or control-heavy code. So CIM is deployed as a specialized accelerator for neural inference, not a general CPU replacement.\n\nRead compute-in-memory through a quant lens rather than a 'faster memory' lens: it moves the operating point on the roofline by driving the bytes-moved term toward zero — the MAC is paid for in physics inside the array instead of in fetched operands — so the figure of merit becomes MACs per joule and per mm² including the ADC/DAC overhead, at the effective bit-precision the analog array can hold. The design question is how much of that data-movement energy you can eliminate before conversion and noise give it back, which is exactly why CIM wins on dense low-precision inference and loses on exact general compute.