CXL (Compute Express Link) Memory is the open standard interconnect protocol that enables cache-coherent memory expansion and sharing across CPUs, GPUs, and memory devices — allowing servers to attach additional memory pools beyond the directly-attached DDR, with CXL memory appearing as regular system memory to applications, addressing the growing gap between compute capacity and memory capacity in AI inference, in-memory databases, and HPC workloads where memory is the primary bottleneck.

Why CXL

• DDR5 channels per CPU: Limited to 8-12 channels → max ~1-2 TB per socket.
• AI inference: Large model weights need more memory than DDR can provide.
• Memory stranding: Some servers underuse memory while others are memory-starved.
• CXL: Attach additional memory devices over PCIe 5.0/6.0 physical layer → expand capacity.

CXL Protocol Types

CXL Device Types

CXL Memory Expander (Type 3)

<svg viewBox="0 0 569 207" xmlns="http://www.w3.org/2000/svg" style="max-width:100%;height:auto" role="img"><rect x="0" y="0" width="569" height="207" rx="12" fill="#0d1117"/><g font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,&quot;Liberation Mono&quot;,monospace" font-size="14"><text xml:space="preserve" x="20" y="31.7"><tspan fill="#c9d1d9"> CPU </tspan><tspan fill="#6e7681">←──</tspan><tspan fill="#c9d1d9">DDR5</tspan><tspan fill="#6e7681">──→</tspan><tspan fill="#c9d1d9"> [Local DRAM: 512 GB]</tspan></text><text xml:space="preserve" x="20" y="50.7"><tspan fill="#c9d1d9">  |</tspan></text><text xml:space="preserve" x="20" y="69.7"><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">├──</tspan><tspan fill="#c9d1d9">CXL 2.0</tspan><tspan fill="#6e7681">──→</tspan><tspan fill="#c9d1d9"> [CXL Memory Expander: 1 TB]</tspan></text><text xml:space="preserve" x="20" y="88.7"><tspan fill="#c9d1d9">  |</tspan></text><text xml:space="preserve" x="20" y="107.7"><tspan fill="#c9d1d9">  </tspan><tspan fill="#6e7681">└──</tspan><tspan fill="#c9d1d9">CXL 2.0</tspan><tspan fill="#6e7681">──→</tspan><tspan fill="#c9d1d9"> [CXL Memory Expander: 1 TB]</tspan></text><text xml:space="preserve" x="20" y="126.7"></text><text xml:space="preserve" x="20" y="145.7"><tspan fill="#c9d1d9"> Total: 2.5 TB addressable memory</tspan></text><text xml:space="preserve" x="20" y="164.7"><tspan fill="#c9d1d9">  - Local DDR: ~80 ns latency, ~400 GB/s BW</tspan></text><text xml:space="preserve" x="20" y="183.7"><tspan fill="#c9d1d9">  - CXL memory: ~150-200 ns latency, ~64-128 GB/s BW per device</tspan></text></g></svg>

CXL Memory Pooling (CXL 2.0+)

  [Server 1] [Server 2] [Server 3]
      \          |          /
       \         |         /
      [CXL Switch / Fabric]
       /    |    |    \    \
   [Mem 1][Mem 2][Mem 3][Mem 4][Mem 5]

• Multiple servers share a pool of CXL memory devices.
• Dynamic allocation: Server 1 gets 2 TB today, server 2 gets 3 TB tomorrow.
• Reduces memory stranding: No more overprovisioning per-server.

Latency and Bandwidth

CXL for AI/ML

• LLM inference: 70B model at FP16 = 140 GB → fits in CXL-expanded memory.
• KV cache expansion: Long context (1M tokens) KV cache in CXL memory → slower but available.
• Recommendation systems: Embedding tables (TBs) in CXL memory pool.
• Tiered memory: Hot data in DDR, warm data in CXL → automatic NUMA-like tiering.

CXL memory is the most significant server architecture evolution since NUMA — by breaking the tight coupling between CPUs and their directly-attached DRAM, CXL enables flexible memory composition that can adapt to workload demands, addressing the memory capacity wall that is increasingly the bottleneck for AI inference and in-memory data processing at scales where adding more DDR channels is physically impossible.