cxl memory
**CXL (Compute Express Link) Memory** is the **open standard interconnect protocol that enables cache-coherent memory expansion and sharing across CPUs, GPUs, and memory devices** — allowing servers to attach additional memory pools beyond the directly-attached DDR, with CXL memory appearing as regular system memory to applications, addressing the growing gap between compute capacity and memory capacity in AI inference, in-memory databases, and HPC workloads where memory is the primary bottleneck.
**Why CXL**
- DDR5 channels per CPU: Limited to 8-12 channels → max ~1-2 TB per socket.
- AI inference: Large model weights need more memory than DDR can provide.
- Memory stranding: Some servers underuse memory while others are memory-starved.
- CXL: Attach additional memory devices over PCIe 5.0/6.0 physical layer → expand capacity.
**CXL Protocol Types**
| Type | Protocol | Purpose | Example |
|------|---------|---------|--------|
| CXL.io | PCIe-compatible | Device discovery, configuration | All CXL devices |
| CXL.cache | Cache coherence | Device caches host memory | Smart NICs, accelerators |
| CXL.mem | Memory access | Host accesses device memory | Memory expanders |
**CXL Device Types**
| Type | CXL Protocols | Use Case |
|------|--------------|----------|
| Type 1 | CXL.io + CXL.cache | Accelerators that cache host memory |
| Type 2 | CXL.io + CXL.cache + CXL.mem | GPUs, FPGAs with own memory |
| Type 3 | CXL.io + CXL.mem | Memory expanders (pure memory) |
**CXL Memory Expander (Type 3)**
```svg
```
**CXL Memory Pooling (CXL 2.0+)**
```
[Server 1] [Server 2] [Server 3]
\ | /
\ | /
[CXL Switch / Fabric]
/ | | \ \
[Mem 1][Mem 2][Mem 3][Mem 4][Mem 5]
```
- Multiple servers share a pool of CXL memory devices.
- Dynamic allocation: Server 1 gets 2 TB today, server 2 gets 3 TB tomorrow.
- Reduces memory stranding: No more overprovisioning per-server.
**Latency and Bandwidth**
| Memory Type | Latency | Bandwidth (per channel) |
|------------|---------|------------------------|
| DDR5 (local) | 70-90 ns | ~50 GB/s per channel |
| CXL 1.1 (direct attach) | 150-250 ns | ~32 GB/s (PCIe 5.0 x8) |
| CXL 2.0 (through switch) | 200-350 ns | ~32 GB/s |
| Remote NUMA (2-socket) | 120-180 ns | ~200 GB/s |
**CXL for AI/ML**
- **LLM inference**: 70B model at FP16 = 140 GB → fits in CXL-expanded memory.
- **KV cache expansion**: Long context (1M tokens) KV cache in CXL memory → slower but available.
- **Recommendation systems**: Embedding tables (TBs) in CXL memory pool.
- **Tiered memory**: Hot data in DDR, warm data in CXL → automatic NUMA-like tiering.
CXL memory is **the most significant server architecture evolution since NUMA** — by breaking the tight coupling between CPUs and their directly-attached DRAM, CXL enables flexible memory composition that can adapt to workload demands, addressing the memory capacity wall that is increasingly the bottleneck for AI inference and in-memory data processing at scales where adding more DDR channels is physically impossible.