NUMA-Aware Optimization is the set of programming and system configuration techniques that account for Non-Uniform Memory Access (NUMA) architecture in multi-socket and modern multi-chiplet systems, where memory access latency and bandwidth depend on the physical distance between the requesting core and the memory controller — a 2-4x performance difference that can dominate application performance if ignored.

Modern servers have 2-8 CPU sockets, each with its own memory controllers and local DRAM. Accessing local memory takes ~80-100ns, while accessing remote memory (through inter-socket interconnects like UPI, Infinity Fabric, or CXL) takes ~150-300ns. Without NUMA awareness, applications may unknowingly place data on remote memory, suffering 2-4x latency and 30-50% bandwidth penalties.

NUMA Architecture:

OS-Level NUMA Management: Linux's numactl and libnuma provide control: membind (allocate memory only on specified nodes), interleave (round-robin allocation across nodes for bandwidth-bound workloads), preferred (try specified node, fall back to others), and cpunodebind (pin threads to specific NUMA nodes). The first-touch policy (default on Linux) allocates memory on the node where the thread first accesses it — this means initialization patterns critically determine data placement.

Application-Level Optimization:

1. Data placement: Allocate data structures on the NUMA node where they'll be most frequently accessed. For partitioned workloads, each thread's data partition should reside on its local node. 2. Thread-data affinity: Pin threads to specific cores and ensure their working data is on the local NUMA node. Use pthread_setaffinity_np() or OpenMP proc_bind(close). 3. NUMA-aware allocation: Use numa_alloc_onnode() or mmap() with MPOL flags for explicit node placement. For large allocations, use huge pages to reduce TLB misses (which are amplified by NUMA latency). 4. Parallel initialization: Initialize data structures in parallel with the same thread mapping that will be used during computation — exploiting first-touch policy for automatic NUMA-local placement. 5. Migration: For workloads with phase-changing access patterns, move_pages() or mbind() can migrate pages between NUMA nodes, though the migration cost (copy + TLB shootdown) must be amortized over subsequent accesses.

NUMA and Shared Data: For data accessed by threads on multiple NUMA nodes, strategies include: replication (maintain per-node copies for read-mostly data), interleaving (spread across nodes for uniform access — sacrifices local latency for balanced bandwidth), and partitioning (decompose shared structures into per-node portions with explicit synchronization).

Measurement: numastat shows per-node allocation statistics; perf stat with NUMA events measures local vs. remote access ratios; Intel VTune and AMD μProf provide visual NUMA locality analysis. Target: >90% local memory access for latency-sensitive workloads.

NUMA-aware optimization is the performance engineering discipline that acknowledges the physical reality of modern parallel hardware — memory is not flat, access is not uniform, and applications that ignore this topology leave 30-60% of potential performance on the table.