CXL 3.0 Memory Pool

Compute Express Link (CXL) 3.0 and 3.1 is the open standard interconnect designed to dismantle the "AI Memory Wall" in modern data centers. Frontier generative AI models, recommendation engines, and multi-tenant cloud workloads require massive memory capacity that far exceeds what can be physically wired directly to CPU memory channels or GPU HBM sites. Furthermore, in traditional server clusters, roughly 25% to 50% of installed DRAM sits stranded — trapped behind CPU sockets that do not need it while memory-starved accelerators crash. CXL 3.0 decouples memory from the processor, enabling disaggregated, dynamically shareable memory pools that can be assigned, carved, and reallocated across clusters of heterogeneous CPUs, GPUs, and custom AI ASICs in real time with hardware-level cache coherency.

The protocol tripartite and the PCIe 6.0 physical layer. CXL 3.0 runs directly over standard PCIe 6.0 electrical physical layers using Pulse Amplitude Modulation 4-level (PAM4) signaling at 64 GT/s, delivering up to 256 GB/s of bidirectional bandwidth over a standard x16 slot. Unlike legacy PCIe, which relies on high-overhead packetized software DMA, CXL multiplexes three distinct protocols across a single link:
1. CXL.io: PCIe-compatible protocol used for device discovery, enumeration, configuration, and standard block DMA transfers.
2. CXL.cache: Enables an accelerator (e.g., GPU or FPGA) to cache system host memory with low-latency, hardware-maintained cache coherency.
3. CXL.mem: Enables the host CPU or GPU to access remote memory on a CXL expander device using standard processor load/store assembly instructions, treating remote DRAM exactly like local physical memory addresses.

The latency hierarchy and the physics of pooled memory. Accessing memory over a serial PCIe/CXL bus incurs an electrical latency penalty compared to direct parallel DDR5 bus lines. Total memory retrieval latency decomposes into four stages:

Where $T_{controller}$ is the memory controller arbitration time, $T_{phy}$ is the serialize/deserialize (SerDes) PHY delay, $T_{switch}$ is the fabric crossbar routing delay, and $T_{media}$ is the DRAM array access ($t_{RCD} + t_{CL}$). Direct-attached DDR5 delivers an access latency of approximately 80 ns. Direct point-to-point CXL.mem adds roughly 70 to 90 ns of SerDes and controller overhead, landing at 150 to 170 ns. Traversing a multi-port CXL 3.0 fabric switch adds a modest 50 ns, providing fully pooled memory at 200 to 240 ns. Crucially, this is an order of magnitude faster than network-based RDMA (RoCE or InfiniBand), which requires 1,500 to 2,500 ns. For large language model (LLM) KV-cache offloading and database indexing, 200 ns is fast enough to run transparently inside the processor's NUMA memory tier.

CXL 3.0 fabric switching and multi-headed coherency. Earlier CXL 1.1 and 2.0 specifications supported only point-to-point connections or simple single-level switching. CXL 3.0 introduces revolutionary fabric capabilities:
- Multi-Headed Devices (MHD): A single CXL memory appliance can connect directly to up to 16 independent host nodes simultaneously, allowing memory blocks to be carved and reassigned without server reboots.
- Hardware-Enforced Memory Sharing: Multiple hosts can not only partition memory, but simultaneously map the *same* physical DRAM pages. CXL 3.0 introduces Back-Invalidation (BI), where the CXL fabric automatically invalidates stale cache lines across all host CPUs when one host writes to shared memory — eliminating expensive software-locking and serialization bottlenecks.
- Peer-to-Peer Accelerator Communication: Multiple GPUs or TPUs can exchange tensors across CXL switches directly without routing traffic through host CPU memory channels.