DRAM
# DRAM Refresh-Bandwidth Tradeoff and Bank-Level Refresh
Every DRAM cell loses charge over time and must be periodically read and rewritten to preserve its data, but a refresh operation makes the memory bank being refreshed unavailable for normal reads and writes during that window — meaning refresh and useful bandwidth are directly competing for the same resource, and the finer-grained the refresh scheme, the less bandwidth gets sacrificed to it at any given moment. The crudest approach, refreshing the entire chip's cells at once, briefly locks out all memory access across the whole device for the duration of that refresh — a significant bandwidth hit if it happens frequently enough, which it must, since cells can only hold charge for a limited time before data would be lost without refreshing. Finer-grained schemes refresh smaller chunks — individual banks or even sub-bank regions — one at a time, so that while one small region is unavailable for refresh, the rest of the memory remains free to service normal traffic.
Bank-level refresh exploits the fact that a DRAM device is internally divided into multiple independent banks, letting the controller refresh one bank while other banks continue serving read and write requests normally — turning what would otherwise be a full-device stall into a much smaller, localized interruption. Because each bank operates largely independently of the others internally, there's no fundamental reason refreshing one bank needs to block access to a different bank — the controller simply has to track which banks are mid-refresh and avoid issuing requests to them until the refresh completes, while continuing to route traffic normally to every other bank. This turns refresh from a periodic full-stop event into a continuously rotating, much smaller tax on available bandwidth.
This same bandwidth-versus-refresh tension is exactly what makes per-die thermal-aware refresh scheduling in an HBM stack worth the engineering effort discussed in an earlier round — a hotter die genuinely needs more frequent refresh, but every extra refresh operation is bandwidth not spent on real traffic, so refreshing every die at the hottest die's rate would waste exactly the bandwidth that bank-level and per-die refresh granularity are designed to recover. The same logic that justifies splitting refresh by bank within a single die applies again across dies in a 3D stack: refreshing conservatively everywhere to cover the worst case costs real throughput, while refreshing only where and when actually needed — at bank granularity within a die, and at per-die granularity within a stack — recovers that throughput without compromising data integrity anywhere it's genuinely at risk.
Micro-bump pitch and spare-TSV redundancy are about how many connections physically exist and how reliably they stay working; refresh-bandwidth tradeoffs are a completely different axis — how efficiently the stack uses the bandwidth those connections provide once they're all working correctly, which is exactly why a well-designed HBM controller has to manage interconnect reliability and refresh scheduling as two genuinely separate engineering problems that both ultimately determine real delivered performance.