hbm stacking
**High Bandwidth Memory (HBM)** is the **3D-stacked DRAM technology that achieves dramatically higher memory bandwidth and energy efficiency than conventional DRAM by vertically stacking multiple DRAM dies interconnected by thousands of Through-Silicon Vias (TSVs)**, placed next to the processor on a silicon interposer — the enabling memory technology for AI accelerators, GPUs, and high-performance computing.
HBM was developed to overcome the "memory wall" — the growing disparity between processor compute capability and memory bandwidth. By going vertical (TSV stacking) and wide (1024-bit bus per stack), HBM achieves bandwidth impossible with traditional package-level interconnects.
**HBM Generations**:
| Generation | Stack | Bandwidth/Stack | Capacity/Stack | Bus Width |
|-----------|-------|-----------------|----------------|----------|
| **HBM** | 4-high | 128 GB/s | 1 GB | 1024-bit |
| **HBM2** | 4-8 high | 256 GB/s | 8 GB | 1024-bit |
| **HBM2E** | 8-high | 460 GB/s | 16 GB | 1024-bit |
| **HBM3** | 8-12 high | 665 GB/s | 24 GB | 1024-bit |
| **HBM3E** | 8-12 high | 1.2 TB/s | 36 GB | 1024-bit |
| **HBM4** | 12-16 high | 1.5+ TB/s | 48+ GB | 2048-bit |
**Architecture**: Each HBM stack consists of a base logic die and multiple DRAM dies interconnected by >5000 TSVs. The base die contains the PHY (physical interface) that communicates with the host processor through microbumps on a silicon interposer. Each stack provides 8 or 16 independent channels with 128-bit data width each, totaling 1024-bit or 2048-bit bus width — versus 64-bit for DDR5. This wide bus achieves high bandwidth at modest per-pin data rates (3.6-9.6 Gbps), keeping power consumption low.
**Interposer Integration**: HBM stacks sit alongside the processor die on a silicon interposer (2.5D integration). The interposer provides high-density wiring (2-4um pitch) impossible with organic package substrates. TSMC's CoWoS (Chip on Wafer on Substrate) and Intel's EMIB (Embedded Multi-die Interconnect Bridge) are the primary interposer technologies. The interposer is a significant cost driver — large interposers for AI chips (800mm²+ reticle limit) require advanced lithography.
**Power Efficiency**: HBM achieves ~3.5-7 pJ/bit — significantly better than DDR5 at ~10-15 pJ/bit. The short, on-interposer signal paths (millimeters vs. centimeters for DDR channels) eliminate the I/O driver power that dominates DDR energy consumption. For AI training (where memory bandwidth directly limits training throughput), HBM's bandwidth-per-watt advantage translates directly to training-efficiency-per-dollar.
**HBM has become the indispensable memory technology for the AI era — every major AI accelerator (NVIDIA H100/B200, AMD MI300, Google TPU, Intel Gaudi) depends on HBM for the memory bandwidth that feeds massive parallel compute engines, establishing HBM as the critical technology linking DRAM innovation to AI performance scaling.**