processing in memory pim design

**Processing-in-Memory (PIM) Chip Architecture: Compute Beside DRAM Arrays — integrating MAC units and logic within DRAM die to eliminate memory bandwidth wall for data-intensive analytics and sparse machine learning** **PIM Core Design Concepts** - **Compute-in-Memory**: MAC operations execute beside DRAM arrays (analog or digital), eliminates PCIe/HBM transfer overhead - **DRAM Layer Integration**: processing logic stacked within memory die or adjacent subarrays, achieves massive parallelism (64k+ operations per cycle) - **Memory Access Pattern Optimization**: algorithms redesigned to maximize data locality, reduce external bandwidth demand **Commercial PIM Architectures** - **Samsung HBM-PIM**: GELU activation, GEMV (generalized matrix-vector multiply) computed in DRAM layer, 3D-stacked HBM integration - **SK Hynix AiMX**: AI-optimized PIM, MAC array per core, interconnect for core-to-core communication - **UPMEM DPU DIMM**: general-purpose processor (DPU: Data Processing Unit) in each DRAM DIMM module, OpenCL-like programming, 256+ DPUs per server **Programming Model and Compilation** - **PIM Intrinsics**: low-level API (memcpy_iop, mram_read) for explicit data movement + compute placement - **OpenCL-like Abstraction**: kernel functions specify computation, automatic offloading to DPU/PIM - **PIM Compiler**: optimizes memory access patterns, tile sizes, pipeline scheduling for PIM constraints - **Challenges**: limited memory per DPU (64 MB MRAM), restricted instruction set, debugging complexity **Applications and Performance Gains** - **Database Analytics**: SELECT + aggregation queries 10-100× faster (bandwidth-limited baseline), no external memory round-trips - **Sparse ML**: sparse matrix operations (pruned neural networks), PIM exploits sparsity efficiently - **Recommendation Systems**: embedding lookups + scoring in-DRAM, recommendation ranking 5-50× speedup - **Bandwidth Wall Elimination**: achieved 1-2 TB/s effective throughput vs ~200 GB/s PCIe Gen4 **Trade-offs and Limitations** - **Limited Compute per DRAM**: ALU set restricted vs GPU, suitable for data movement bottleneck, not compute bottleneck - **Programmability vs Efficiency**: high-level API simpler but loses PIM-specific optimization opportunities - **Data Movement Still Exists**: DPU-to-CPU communication adds latency, not all workloads benefit **Future Roadmap**: PIM expected as standard in server DRAM, specialized for ML inference + analytics, complementary to GPU (GPU for compute-heavy, PIM for memory-heavy).