A memory controller is the hardware that translates processor and accelerator requests into legal, efficient command sequences for external DRAM. It accepts reads, writes, atomics, or cache-line transactions; maps addresses into channels, ranks, bank groups, banks, rows, and columns; schedules commands around timing constraints; refreshes cells; trains the physical interface; and returns data in the required order. For AI and high-performance computing, controller policy often determines how much of expensive DDR, LPDDR, or HBM bandwidth software can actually use.

DRAM is a two-dimensional analog array hidden behind a digital command interface. Accessing a row activates thousands of cells into local sense amplifiers, turning the row buffer into a fast temporary store. A column command then transfers selected data. Access to another row in the same bank requires precharge and a new activate. Controllers exploit parallel banks and row hits while obeying minimum delays for activation, sensing, restoration, bus turnaround, and power delivery.

Address mapping controls both parallelism and locality. Consecutive cache lines may be striped across channels to distribute bandwidth, across banks to overlap operations, or kept within a row to maximize row-buffer hits. XOR hashing reduces power-of-two hotspots but makes performance less intuitive. Page allocation, tensor layout, and controller mapping interact: an unfortunate stride can repeatedly select one bank while the rest are idle. Documentation or performance counters are essential for software optimization.

<svg viewBox="0 0 960 380" xmlns="http://www.w3.org/2000/svg" role="img" aria-labelledby="mcTitle mcDesc">
  <title id="mcTitle">Memory controller request and scheduling pipeline</title>
  <desc id="mcDesc">CPU and accelerator requests enter queues, are mapped and scheduled, pass through a PHY, and access parallel DRAM banks.</desc>
  <rect width="960" height="380" rx="18" fill="#0b1017"/>
  <text x="480" y="32" text-anchor="middle" fill="#f1f5f9" font-size="20" font-weight="700">Turn irregular requests into efficient DRAM commands</text>
  <defs><marker id="mcArrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto"><path d="M0 0L10 5L0 10Z" fill="#60a5fa"/></marker></defs>
  <g fill="#111c2b" stroke="#60a5fa" stroke-width="2"><rect x="30" y="95" width="130" height="105" rx="10"/><rect x="215" y="75" width="220" height="145" rx="10"/><rect x="490" y="95" width="120" height="105" rx="10"/><rect x="665" y="58" width="260" height="222" rx="10"/></g>
  <g fill="#dbeafe" text-anchor="middle" font-size="13" font-weight="600"><text x="95" y="130">CPU / GPU</text><text x="95" y="151">DMA requests</text><text x="325" y="106">Memory controller</text><text x="325" y="137">Read/write queues</text><text x="325" y="161">Address map + ECC</text><text x="325" y="185">Timing scheduler</text><text x="550" y="132">PHY</text><text x="550" y="154">training</text><text x="795" y="86">DRAM channel</text></g>
  <g stroke="#60a5fa" stroke-width="3" fill="none" marker-end="url(#mcArrow)"><path d="M160 147H208"/><path d="M435 147H483"/><path d="M610 147H658"/></g>
  <g fill="#172554" stroke="#34d399" stroke-width="1.5"><rect x="690" y="110" width="90" height="54" rx="7"/><rect x="810" y="110" width="90" height="54" rx="7"/><rect x="690" y="190" width="90" height="54" rx="7"/><rect x="810" y="190" width="90" height="54" rx="7"/></g>
  <g fill="#bbf7d0" text-anchor="middle" font-size="12"><text x="735" y="142">Bank 0</text><text x="855" y="142">Bank 1</text><text x="735" y="222">Bank 2</text><text x="855" y="222">Bank N</text></g>
  <path d="M325 225V286H795V280" fill="none" stroke="#f59e0b" stroke-width="2" stroke-dasharray="7 5" marker-end="url(#mcArrow)"/>
  <text x="325" y="314" text-anchor="middle" fill="#fbbf24" font-size="13">Refresh, QoS, power states, telemetry</text>
  <text x="480" y="352" text-anchor="middle" fill="#94a3b8" font-size="13">Useful bandwidth depends on bank parallelism, row locality, bus direction, and timing legality.</text>
</svg>

Scheduling is a constrained optimization performed every cycle. A common policy prioritizes ready row hits, then older requests, often called first-ready first-come-first-served. It improves throughput by avoiding activate and precharge delays, but an endless stream of hits can starve a miss. Controllers add age thresholds, per-client credits, deadlines, and write-drain rules. Real-time agents may reserve service while CPUs and accelerators share remaining bandwidth.

Read and write buses are bidirectional and expensive to turn around. Writes are buffered so the controller can serve latency-sensitive reads, then drained in batches to amortize direction changes. Waiting too long risks a full write queue and backpressure; draining too often wastes data-bus cycles. Read-after-write hazards require forwarding or ordering. Partial writes may trigger read-modify-write unless byte masks and ECC organization support them directly.

DRAM timing rules encode device physics and power limits. Constraints such as row-to-column delay, precharge time, row cycle time, activate-to-activate spacing, and four-activate windows prevent incomplete sensing or excessive simultaneous current. Bank-group rules may allow faster commands to different groups than within one group. The controller maintains counters or reservation calendars so it never issues an illegal sequence, including at frequency changes and temperature-dependent modes.

Refresh preserves charge that leaks from DRAM cells. Conventional all-bank or per-bank refresh periodically blocks resources. Fine-granularity modes trade command frequency against pause length. Temperature can increase refresh demand, while retention-aware research attempts to avoid refreshing strong cells unnecessarily. Controllers postpone or pull in refresh within specification to place it in idle windows, but heavy traffic eventually pays the cost. HBM and advanced devices add row-hammer mitigation and repair behaviors that also consume bandwidth.

Theoretical bandwidth is easy to calculate and hard to sustain. If a channel transfers \(w\) bytes per edge at effective rate \(r\), peak bandwidth is \(B=w r\). Delivered bandwidth subtracts refresh, command bubbles, reads-to-writes turnaround, row conflicts, protocol overhead, and imbalance. Efficiency may be high for long sequential bursts and much lower for random small accesses. Reporting only peak bandwidth hides the controller and workload behaviors that determine tokens per second.

HBM changes scale more than basic principles. A stack exposes many independent channels or pseudo-channels through a very wide die-to-die interface. The controller can service extraordinary aggregate bandwidth, but only if requests distribute across those resources. Hundreds of queues, timing trackers, and ECC paths consume area and power. Thermal conditions in the stack, lane repair, pseudo-channel mapping, and package test access add concerns absent from a conventional DIMM.

AI workloads create both regular streams and pathological contention. Matrix tiles can generate long predictable bursts, while embedding lookups, sparse models, attention KV caches, and mixture-of-experts routing produce irregular accesses. Prefetchers and DMA engines should coalesce transactions and align them to bursts. Memory-side compression can save bandwidth when data is compressible but adds latency and metadata. Controller counters should reveal channel balance, row-hit rate, queue occupancy, stall source, and achieved bytes per cycle.

QoS is necessary when several agents share memory. A display or network port cannot miss a service deadline because an accelerator launched a bulk transfer. Pure priority can starve low classes; pure fairness can violate latency. Token buckets, weighted arbitration, maximum-latency overrides, bandwidth reservations, and traffic shaping at sources are combined. The controller must account for bank conflicts: accepting a request is not the same as guaranteeing the cycle in which its data returns.

Reliability, availability, and serviceability are integrated into the data path. ECC commonly corrects single-bit errors and detects stronger patterns; server systems may use symbol-based schemes, memory mirroring, sparing, patrol scrubbing, and poison propagation. Error addresses and syndromes are logged for diagnosis. Scrub traffic competes with normal requests, while correction and retry affect latency. Security features may encrypt memory and protect integrity, adding tweak generation, metadata access, or replay defenses.

The controller and PHY cooperate during initialization and training. They discover device geometry, configure mode registers, calibrate impedance, align strobes, center sampling windows, and compensate per-bit skew. DDR modules may require topology-specific leveling; HBM uses short package wires but thousands of lanes. Training must work across process, voltage, temperature, and aging, recover after low-power modes, and expose actionable failure status during board bring-up.

Power management spans clock gating, self-refresh, power-down states, and dynamic frequency changes. Closing rows can save background power but sacrifices future hits. Low-power entry saves energy only if the idle interval exceeds entry and exit overhead. LPDDR adds aggressive modes for mobile systems; servers favor readiness and reliability. A good policy forecasts idleness without introducing tail-latency spikes, and coordinates with operating-system or firmware expectations.

Verification requires far more than read and write correctness. Constrained-random traffic stresses timing boundaries, queue full conditions, refresh collisions, rank changes, ECC faults, reset, and frequency transitions. Assertions prove no illegal DRAM command is issued and no accepted request is lost or duplicated. Reference models check ordering and data. Performance regression suites use synthetic and application traces because a logically correct scheduler can silently lose substantial throughput after a minor priority change.

Memory-controller design is ultimately system co-design. Cache policy, NoC routing, page allocation, tensor layout, PHY capability, package wiring, DRAM organization, firmware, and thermal limits all shape observed service. The strongest controller is not the one with the cleverest isolated scheduler; it is the one that delivers predictable useful bandwidth, latency, fairness, data integrity, and power efficiency under the actual workload mix.