Computational In-Memory (CIM) and Processing-In-Memory (PIM) is the semiconductor architecture paradigm that performs computation directly within or adjacent to memory arrays — eliminating the von Neumann bottleneck where data must be transferred between separate memory and processing units, achieving 10-100× improvement in energy efficiency for data-intensive workloads like neural network inference by performing multiply-accumulate (MAC) operations using the physical properties of the memory elements themselves.

The Memory Wall Problem

<svg viewBox="0 0 527 226" xmlns="http://www.w3.org/2000/svg" style="max-width:100%;height:auto" role="img"><rect x="0" y="0" width="527" height="226" rx="12" fill="#0d1117"/><g font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,&quot;Liberation Mono&quot;,monospace" font-size="14"><text xml:space="preserve" x="20" y="31.7"><tspan fill="#c9d1d9">Von Neumann architecture:</tspan></text><text xml:space="preserve" x="20" y="50.7"><tspan fill="#c9d1d9">  [Processor] </tspan><tspan fill="#6e7681">←──</tspan><tspan fill="#c9d1d9"> data bus </tspan><tspan fill="#6e7681">──→</tspan><tspan fill="#c9d1d9"> [Memory]</tspan></text><text xml:space="preserve" x="20" y="69.7"><tspan fill="#c9d1d9">  Compute: ~1 pJ/operation</tspan></text><text xml:space="preserve" x="20" y="88.7"><tspan fill="#c9d1d9">  Data movement: ~100-1000 pJ/access</tspan></text><text xml:space="preserve" x="20" y="107.7"><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">→</tspan><tspan fill="#c9d1d9"> 99% of energy spent on data movement, not computation!</tspan></text><text xml:space="preserve" x="20" y="126.7"></text><text xml:space="preserve" x="20" y="145.7"><tspan fill="#c9d1d9">CIM architecture:</tspan></text><text xml:space="preserve" x="20" y="164.7"><tspan fill="#c9d1d9">  [Memory + Compute combined]</tspan></text><text xml:space="preserve" x="20" y="183.7"><tspan fill="#c9d1d9">  MAC inside memory array: ~1-10 pJ total</tspan></text><text xml:space="preserve" x="20" y="202.7"><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">→</tspan><tspan fill="#c9d1d9"> 10-100× energy reduction for neural network inference</tspan></text></g></svg>

CIM Approaches

Analog CIM for Neural Networks

<svg viewBox="0 0 645 321" xmlns="http://www.w3.org/2000/svg" style="max-width:100%;height:auto" role="img"><rect x="0" y="0" width="645" height="321" rx="12" fill="#0d1117"/><g font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,&quot;Liberation Mono&quot;,monospace" font-size="14"><text xml:space="preserve" x="20" y="31.7"><tspan fill="#c9d1d9">Matrix-Vector Multiply (key neural network operation):</tspan></text><text xml:space="preserve" x="20" y="50.7"><tspan fill="#c9d1d9">  y = W × x</tspan></text><text xml:space="preserve" x="20" y="69.7"></text><text xml:space="preserve" x="20" y="88.7"><tspan fill="#c9d1d9">In CIM (crossbar array):</tspan></text><text xml:space="preserve" x="20" y="107.7"><tspan fill="#c9d1d9">  - Weights W stored as conductance values in memory cells</tspan></text><text xml:space="preserve" x="20" y="126.7"><tspan fill="#c9d1d9">  - Input x applied as voltages to wordlines</tspan></text><text xml:space="preserve" x="20" y="145.7"><tspan fill="#c9d1d9">  - Output current I = Σ(G_ij × V_i) </tspan><tspan fill="#6e7681">→</tspan><tspan fill="#c9d1d9"> Kirchhoff's current law does MAC!</tspan></text><text xml:space="preserve" x="20" y="164.7"><tspan fill="#c9d1d9">  - ADC converts summed current to digital output</tspan></text><text xml:space="preserve" x="20" y="183.7"></text><text xml:space="preserve" x="20" y="202.7"><tspan fill="#c9d1d9">  V₁ </tspan><tspan fill="#6e7681">──┬─</tspan><tspan fill="#c9d1d9">[G₁₁]</tspan><tspan fill="#6e7681">─┬─</tspan><tspan fill="#c9d1d9">[G₁₂]</tspan><tspan fill="#6e7681">─┬─</tspan><tspan fill="#c9d1d9"> </tspan><tspan fill="#6e7681">→</tspan><tspan fill="#c9d1d9"> I₁ = G₁₁V₁ + G₂₁V₂</tspan></text><text xml:space="preserve" x="20" y="221.7"><tspan fill="#c9d1d9">       </tspan><tspan fill="#6e7681">│</tspan><tspan fill="#c9d1d9">        </tspan><tspan fill="#6e7681">│</tspan><tspan fill="#c9d1d9">        </tspan><tspan fill="#6e7681">│</tspan></text><text xml:space="preserve" x="20" y="240.7"><tspan fill="#c9d1d9">  V₂ </tspan><tspan fill="#6e7681">──┴─</tspan><tspan fill="#c9d1d9">[G₂₁]</tspan><tspan fill="#6e7681">─┴─</tspan><tspan fill="#c9d1d9">[G₂₂]</tspan><tspan fill="#6e7681">─┴─</tspan><tspan fill="#c9d1d9"> </tspan><tspan fill="#6e7681">→</tspan><tspan fill="#c9d1d9"> I₂ = G₁₂V₁ + G₂₂V₂</tspan></text><text xml:space="preserve" x="20" y="259.7"></text><text xml:space="preserve" x="20" y="278.7"><tspan fill="#c9d1d9">  Single clock cycle: Entire matrix-vector multiply!</tspan></text><text xml:space="preserve" x="20" y="297.7"><tspan fill="#c9d1d9">  O(1) time instead of O(N²) operations</tspan></text></g></svg>

SRAM CIM (Digital/Near-Digital)

• TSMC SRAM CIM: Modified 6T SRAM bitcell with additional compute transistors.
• Perform bit-serial multiplication within SRAM macro.
• Advantage: Uses existing SRAM technology, digital precision.
• Used in: Edge AI accelerators, IoT inference chips.

Performance Comparison

Challenges

Industry Status

Computational in-memory is the paradigm shift that addresses the fundamental energy bottleneck of the von Neumann architecture — by performing computation where data lives rather than moving data to where computation happens, CIM chips achieve orders-of-magnitude improvement in energy efficiency for AI inference, making them the most promising architecture for deploying neural networks at the edge where every millijoule of energy matters.