Computational In-Memory (CIM) and Processing-In-Memory (PIM) is the semiconductor architecture paradigm that performs computation directly within or adjacent to memory arrays — eliminating the von Neumann bottleneck where data must be transferred between separate memory and processing units, achieving 10-100× improvement in energy efficiency for data-intensive workloads like neural network inference by performing multiply-accumulate (MAC) operations using the physical properties of the memory elements themselves.
The Memory Wall Problem
Von Neumann architecture:
[Processor] ←── data bus ──→ [Memory]
Compute: ~1 pJ/operation
Data movement: ~100-1000 pJ/access
→ 99% of energy spent on data movement, not computation!
CIM architecture:
[Memory + Compute combined]
MAC inside memory array: ~1-10 pJ total
→ 10-100× energy reduction for neural network inference
CIM Approaches
| Approach | Memory Type | Compute Method | Maturity |
|---|---|---|---|
| SRAM CIM | SRAM bitcell | Digital/analog MAC in array | Production (TSMC, Samsung) |
| ReRAM CIM | Resistive RAM | Analog current-mode MAC | R&D/Pilot |
| Flash CIM | NOR flash | Analog current summation | Production (some) |
| MRAM CIM | STT-MRAM | Resistance-based MAC | Research |
| DRAM PIM | HBM/GDDR with logic | Digital compute near memory | Production (Samsung HBM-PIM) |
Analog CIM for Neural Networks
Matrix-Vector Multiply (key neural network operation):
y = W × x
In CIM (crossbar array):
- Weights W stored as conductance values in memory cells
- Input x applied as voltages to wordlines
- Output current I = Σ(G_ij × V_i) → Kirchhoff's current law does MAC!
- ADC converts summed current to digital output
V₁ ──┬─[G₁₁]─┬─[G₁₂]─┬─ → I₁ = G₁₁V₁ + G₂₁V₂
│ │ │
V₂ ──┴─[G₂₁]─┴─[G₂₂]─┴─ → I₂ = G₁₂V₁ + G₂₂V₂
Single clock cycle: Entire matrix-vector multiply!
O(1) time instead of O(N²) operations
SRAM CIM (Digital/Near-Digital)
- TSMC SRAM CIM: Modified 6T SRAM bitcell with additional compute transistors.
- Perform bit-serial multiplication within SRAM macro.
- Advantage: Uses existing SRAM technology, digital precision.
- Used in: Edge AI accelerators, IoT inference chips.
Performance Comparison
| Platform | ResNet-50 Inference | Energy/Inference |
|---|---|---|
| GPU (A100) | 0.1 ms | ~10 mJ |
| Digital accelerator (TPU) | 0.2 ms | ~2 mJ |
| SRAM CIM chip | 0.5 ms | ~0.2 mJ |
| ReRAM CIM chip | 1 ms | ~0.05 mJ |
Challenges
| Challenge | Issue | Status |
|---|---|---|
| ADC overhead | ADC conversion dominates energy in analog CIM | Multi-bit ADC design |
| Precision | Analog compute limited to 4-8 bit precision | Acceptable for inference |
| Variability | Memory cell variations → compute errors | Calibration, training-aware |
| Write endurance | ReRAM limited write cycles | Read-mostly inference OK |
| Programming | Must map NN weights to memory array | Compiler/mapper tools |
Industry Status
| Company | Approach | Product |
|---|---|---|
| TSMC | SRAM CIM macro | Available to customers (N7, N5) |
| Samsung | HBM-PIM | Deployed in HPC systems |
| IBM | PCM-based CIM | Analog AI research chip |
| Mythic | Flash-based CIM | M1076 edge AI chip |
| Envision | SRAM CIM | Edge AI SoC |
Computational in-memory is the paradigm shift that addresses the fundamental energy bottleneck of the von Neumann architecture — by performing computation where data lives rather than moving data to where computation happens, CIM chips achieve orders-of-magnitude improvement in energy efficiency for AI inference, making them the most promising architecture for deploying neural networks at the edge where every millijoule of energy matters.
Explore 500+ Semiconductor & AI Topics
From EUV lithography to CUDA optimization — search the full knowledge base or chat with our AI assistant.