numa aware optimization
**NUMA-Aware Optimization** is the **set of programming and system configuration techniques that account for Non-Uniform Memory Access (NUMA) architecture in multi-socket and modern multi-chiplet systems**, where memory access latency and bandwidth depend on the physical distance between the requesting core and the memory controller — a 2-4x performance difference that can dominate application performance if ignored.
Modern servers have 2-8 CPU sockets, each with its own memory controllers and local DRAM. Accessing local memory takes ~80-100ns, while accessing remote memory (through inter-socket interconnects like UPI, Infinity Fabric, or CXL) takes ~150-300ns. Without NUMA awareness, applications may unknowingly place data on remote memory, suffering 2-4x latency and 30-50% bandwidth penalties.
**NUMA Architecture**:
| Component | Local | Remote | Impact |
|-----------|-------|--------|--------|
| **Memory latency** | 80-100ns | 150-300ns | 2-3x slower |
| **Memory bandwidth** | 100% | 50-70% | Throughput limited |
| **Interconnect** | N/A | UPI/IF/CXL links | Shared, congestion-prone |
| **Cache coherence** | L3 hit ~10ns | Remote L3 snoop ~60-100ns | Directory overhead |
**OS-Level NUMA Management**: Linux's **numactl** and **libnuma** provide control: **membind** (allocate memory only on specified nodes), **interleave** (round-robin allocation across nodes for bandwidth-bound workloads), **preferred** (try specified node, fall back to others), and **cpunodebind** (pin threads to specific NUMA nodes). The **first-touch policy** (default on Linux) allocates memory on the node where the thread first accesses it — this means initialization patterns critically determine data placement.
**Application-Level Optimization**:
1. **Data placement**: Allocate data structures on the NUMA node where they'll be most frequently accessed. For partitioned workloads, each thread's data partition should reside on its local node.
2. **Thread-data affinity**: Pin threads to specific cores and ensure their working data is on the local NUMA node. Use `pthread_setaffinity_np()` or OpenMP `proc_bind(close)`.
3. **NUMA-aware allocation**: Use `numa_alloc_onnode()` or `mmap()` with MPOL flags for explicit node placement. For large allocations, use huge pages to reduce TLB misses (which are amplified by NUMA latency).
4. **Parallel initialization**: Initialize data structures in parallel with the same thread mapping that will be used during computation — exploiting first-touch policy for automatic NUMA-local placement.
5. **Migration**: For workloads with phase-changing access patterns, `move_pages()` or `mbind()` can migrate pages between NUMA nodes, though the migration cost (copy + TLB shootdown) must be amortized over subsequent accesses.
**NUMA and Shared Data**: For data accessed by threads on multiple NUMA nodes, strategies include: **replication** (maintain per-node copies for read-mostly data), **interleaving** (spread across nodes for uniform access — sacrifices local latency for balanced bandwidth), and **partitioning** (decompose shared structures into per-node portions with explicit synchronization).
**Measurement**: **numastat** shows per-node allocation statistics; **perf stat** with NUMA events measures local vs. remote access ratios; Intel VTune and AMD μProf provide visual NUMA locality analysis. Target: >90% local memory access for latency-sensitive workloads.
**NUMA-aware optimization is the performance engineering discipline that acknowledges the physical reality of modern parallel hardware — memory is not flat, access is not uniform, and applications that ignore this topology leave 30-60% of potential performance on the table.**