memory bandwidth high
**High Bandwidth Memory (HBM)** is the **3D-stacked DRAM architecture that delivers 10–20× more memory bandwidth than conventional DDR by stacking multiple DRAM dies vertically and connecting them with through-silicon vias (TSVs) to a logic die** — enabling AI accelerators, GPUs, and HPC processors to feed their compute units at multi-terabyte-per-second rates that would be physically impossible with wide-bus DDR or LPDDR interfaces.
**HBM Architecture**
```svg
```
- **TSV density**: Thousands of vertical connections per stack (1024–2048 I/O per stack).
- **Interface width**: 1024-bit bus per stack (vs 64-bit for DDR5).
- **Stack height**: 4–12 DRAM dies per stack.
**HBM Generation Comparison**
| Generation | Year | BW/Stack | Capacity/Stack | I/O Pins | Voltage |
|-----------|------|----------|----------------|----------|---------|
| HBM1 | 2015 | 128 GB/s | 1–2 GB | 1024 | 1.2 V |
| HBM2 | 2016 | 256 GB/s | 4–8 GB | 1024 | 1.2 V |
| HBM2E | 2019 | 460 GB/s | 8–16 GB | 1024 | 1.2 V |
| HBM3 | 2022 | 819 GB/s | 16–24 GB | 1024 | 1.1 V |
| HBM3E | 2024 | 1.2 TB/s | 24–36 GB | 1024 | 1.1 V |
**Manufacturing Process**
- **DRAM die fabrication**: Standard DRAM process (1z/1α/1β nm class) with TSV integration.
- **TSV formation**: Etch → liner deposition → barrier/seed → copper fill → CMP → TSV reveal by backside thinning.
- **Die thinning**: Each DRAM die thinned to ~30–50 µm before stacking.
- **Micro-bump bonding**: Cu-pillar micro-bumps with 55 µm pitch (HBM2) → 40 µm (HBM3) connecting dies.
- **Mass reflow or thermocompression bonding** for stack assembly.
- **Underfill**: Capillary underfill injected to mechanically stabilize stack.
**Integration with Logic Die**
- **2.5D (CoWoS)**: HBM stack and logic die placed side-by-side on a silicon interposer → TSVs in interposer carry signals between them. Used in NVIDIA H100, AMD MI300.
- **3D stacking**: HBM placed directly on top of logic die (less common due to thermal concerns).
- **Active interposer**: Interposer contains routing + some logic elements.
**Thermal Challenges**
- DRAM generates heat proportional to bandwidth × access frequency.
- Heat must flow laterally through interposer or out the top of the stack.
- HBM3E operating at full bandwidth can dissipate 10–15 W per stack.
- Mitigation: TIM (thermal interface material) on top die, heat spreader, liquid cooling.
**Applications**
| System | HBM Generation | Stacks | Total BW |
|--------|---------------|--------|----------|
| NVIDIA A100 | HBM2E | 5 | 2.0 TB/s |
| NVIDIA H100 | HBM3 | 5 | 3.35 TB/s |
| AMD MI300X | HBM3 | 8 | 5.2 TB/s |
| Google TPU v5 | HBM3 | Varies | 4.8 TB/s |
| Intel Gaudi 3 | HBM2E | 6 | 3.7 TB/s |
HBM is **the bandwidth solution that makes modern AI training and inference economically viable** — its combination of extreme bandwidth, low power per bit, and compact footprint on an interposer has become the de facto memory standard for high-performance AI accelerators, with every major AI chip design now centered around maximizing effective HBM utilization.