semiconductor cim

**Compute-in-memory (CIM) and processing-in-memory (PIM)** are semiconductor architecture paradigms that perform arithmetic operations inside or directly beside the memory array, eliminating the dominant energy and latency cost of moving data between memory and a separate processor. In conventional Von Neumann architectures, 60-70% of inference energy is consumed reading neural network weights from DRAM — CIM/PIM attacks this directly. ```svg Compute-In-Memory / Processing-In-Memory (CIM/PIM) eliminate the memory wall by performing MAC operations inside or beside the memory array — data movement is the dominant energy cost Memory Wall Problem Conventional (Von Neumann): SRAM weights bitcells data bus 70% energy ALU / MAC compute Bottleneck: weights travel memory->bus->ALU every inference step — bandwidth wall CIM (compute-in-memory): SRAM CIM array MAC inside array BL current = W x A result only No weight data movement — partial sums accumulate on bitlines — 10-100x lower energy CIM / PIM Taxonomy Near-Memory PIM Logic die stacked on/beside DRAM (HBM-PIM, UPMEM) ~1000x bandwidth vs off-chip; digital compute; ISA-controlled SRAM CIM — Analog Bitline current = sum(W x A); ADC reads result High TOPS/W for inference; limited precision (4-8 bit) SRAM CIM — Digital Digital accumulator in bitcell peripheral — deterministic Lower TOPS/W than analog but no ADC noise; easier STA NVM CIM (RRAM/PCM/Flash): weights stored non-volatile — emerging; IBM, Mythic use Flash-based analog MAC AI Inference Energy Breakdown Conventional NN inference energy sources: DRAM access (weight reads) ~60-70% On-chip SRAM ~15% MAC compute ~10% Other ~5% CIM/PIM eliminates the 60-70% DRAM term: TOPS/W Comparison (INT4 inference) GPU (H100): ~1-3 TOPS/W (memory-bound) SRAM CIM chip: ~10-100 TOPS/W (data stays in array) Precision vs energy trade-off: INT8 → INT4 → binary lower precision → less ADC resolution needed → less area + power Key Challenges ADC cost: each bitline needs an ADC — area-intensive Analog noise: kT/C, mismatch, supply variation limit precision Write endurance: NVM CIM limits retraining (Flash ~1e5 cycles) Programming: non-standard ISA, no GPU-style ecosystem yet Weight update: analog CIM not suited for on-chip training Thermal: MAC inside array raises local bitcell temperature Industry Products Samsung HBM-PIM (Aquabolt-PIM): DRAM bank + programmable core on logic die — 2x BW, AI workloads Mythic M1108 AMP: Flash-based analog CIM — 25 TOPS, 3W — fixed-weight inference only UPMEM / Micron HMC-PIM: DRAM rows + RISC cores — genomics, DB, sparse ML graphs SRAM CIM Design Pattern Activate word-line → all bitcells in row drive bitline simultaneously Bitline charge = weighted sum (charge-domain MAC) Multi-row partial sums → analog adder → single ADC per column Input activation broadcasts over word-lines; weights fixed in cells Result: 128-256 MACs per column in parallel — no data bus needed MIT, Stanford, TSMC, and ARM all publish SRAM CIM silicon results 60-70% energy in DRAM weight reads (baseline) 10-100x TOPS/W CIM vs GPU (INT4 inference) Samsung HBM-PIM 2x effective bandwidth for AI ADC = area bottleneck 1 ADC per bitline column 4-8 bit precision typical analog CIM resolution CIM/PIM flips the Von Neumann bottleneck: instead of moving data to the processor, move the computation to the data — memory becomes the accelerator Critical for edge AI inference (Syntiant, Mythic) and HPC memory scaling (Samsung, SK Hynix PIM roadmaps) ``` **The memory wall is the fundamental problem.** In autoregressive LLM inference and most edge AI workloads, the compute units are starved waiting for weight data. DRAM bandwidth is limited by off-chip signaling and pin count; on-chip SRAM bandwidth is limited by array size. The Von Neumann model requires every weight to traverse the memory bus to the ALU on every inference step, regardless of how fast the arithmetic can run. CIM eliminates this bus entirely by making the memory array itself perform the multiply-accumulate (MAC) operation. **SRAM CIM works by performing analog computation on the bitlines.** In a conventional SRAM read, activating a word-line drives all bitcells in a row onto shared bitlines, and a sense amplifier detects the bit value. In analog CIM, multiple rows are activated simultaneously with partial word-line voltages corresponding to input activation values. Each bitcell contributes a current proportional to its stored weight times the applied voltage. The bitline current sums these contributions — performing a dot product in the physics of the wire, not in a separate multiplier. An ADC at the bottom of each bitline column converts the analog sum to a digital partial result. **Digital CIM variants** place a small digital accumulator in the periphery of each bitcell column rather than using analog current summation. This avoids ADC noise and the analog precision limitations but sacrifices some energy efficiency. MIT, Stanford, TSMC, and ARM have all published silicon results for both approaches, with digital CIM achieving 10-100 TOPS/W at INT4 precision versus 1-3 TOPS/W for a GPU doing the same inference task. **Near-memory PIM** takes a different path: rather than computing inside the array, it places a digital processor die on top of or beside the DRAM stack. Samsung's HBM-PIM (Aquabolt-PIM) adds a programmable AI core on the logic die of an HBM stack, giving the processor access to DRAM bandwidth without the off-chip bottleneck. UPMEM places RISC cores inside DRAM rows, targeting genomics and database workloads that are purely memory-bandwidth-limited. **Key challenges** limit deployment. Analog CIM precision tops out at 4-8 bits, which is adequate for INT4/INT8 inference but not for FP16 training. Each bitline column requires an ADC, and ADC arrays are area-intensive. NVM-based CIM (Flash, RRAM, PCM) stores weights non-volatitely — eliminating DRAM access entirely — but write endurance limits in-situ retraining and analog device variability reduces effective precision. The programming model is non-standard: existing GPU software stacks cannot directly target CIM hardware. **Commercial CIM products** include Mythic's M1108 AMP (Flash-based analog MAC, 25 TOPS at 3W), Syntiant's NDP (SRAM CIM for always-on keyword detection at sub-mW), and IBM's phase-change memory CIM research chips for transformer inference. Samsung and SK Hynix both have PIM roadmaps targeting HBM4 and beyond.

Go deeper with CFSGPT

Get AI-powered deep-dives, save terms, and run advanced simulations — free account.

Create Free Account