hbm memory
**High Bandwidth Memory (HBM)** is a **3D-stacked DRAM architecture that vertically stacks 4-16 DRAM dies connected by thousands of through-silicon vias (TSVs) on a silicon base die, co-packaged with a processor on a silicon interposer** — delivering memory bandwidth of 1-2+ TB/s per stack, 3-5× the bandwidth of traditional GDDR, making HBM the essential memory technology for AI training accelerators, HPC, and high-performance networking.
**HBM Architecture:**
```svg
```
**HBM Generations:**
| Generation | Stacks | BW/Stack | Capacity/Stack | Key Feature |
|-----------|--------|----------|---------------|-------------|
| HBM1 (2013) | 4-high | 128 GB/s | 1 GB | First TSV DRAM |
| HBM2 (2016) | 4-8 high | 256 GB/s | 8 GB | Doubled density |
| HBM2E (2020) | 8-high | 460 GB/s | 16 GB | 3.6 Gbps/pin |
| HBM3 (2022) | 8-12 high | 819 GB/s | 24 GB | Independent channels |
| HBM3E (2024) | 8-12 high | 1.18 TB/s | 36 GB | 9.6 Gbps/pin |
| HBM4 (~2026) | 12-16 high | 1.5+ TB/s | 48 GB+ | Wider interface |
**Key to Bandwidth:**
HBM achieves high bandwidth through massive parallelism, not high clock speed:
- **1024-bit wide interface** per stack (vs. 32-bit for DDR5)
- Multiple **independent channels** (16 channels in HBM3/3E, each 64-bit)
- Moderate data rate per pin (6.4-9.6 Gbps vs. DDR5's 4.8-8.4 Gbps)
- BW = 1024 bits × data_rate = 1024 × 9.6 Gbps = 1.18 TB/s (HBM3E)
**Manufacturing:**
- **TSV formation**: Deep reactive-ion etch (DRIE) through ~50μm thinned DRAM die, Cu-filled, <10μm diameter
- **Die thinning**: Grinding + CMP to ~40-50μm per die (from standard ~750μm)
- **Stacking**: Thermocompression bonding, die-to-die alignment <1μm
- **Mass reflow**: Microbumps (Cu pillar + SnAg solder, 25-36μm pitch) flow at ~260°C
- **Underfill**: Capillary or non-conductive film (NCF) between die for mechanical support
- Only SK Hynix, Samsung, and Micron manufacture HBM; SK Hynix leads market share
**AI Accelerator HBM Configuration:**
| Accelerator | HBM Type | Stacks | Total BW | Total Capacity |
|------------|----------|--------|---------|---------------|
| NVIDIA H100 | HBM3 | 5 | 3.35 TB/s | 80 GB |
| NVIDIA H200 | HBM3E | 6 | 4.8 TB/s | 141 GB |
| NVIDIA B200 | HBM3E | 8 | 8 TB/s | 192 GB |
| AMD MI300X | HBM3 | 8 | 5.3 TB/s | 192 GB |
**Challenges**: TSV yield (one bad die in a 12-high stack = scrapped stack), thermal management (thinned die have less thermal mass; DRAM junction temperature must stay <95°C), and cost ($20-30+ per GB vs $2-3 for DDR5, supply-constrained).
**HBM is the memory technology that makes modern AI training possible** — without HBM's terabyte-per-second bandwidth enabling GPUs to feed their thousands of compute cores, the massive parallelism of AI accelerators would be starved for data, making HBM the most critical enabling component in the AI hardware ecosystem.