numa aware programming optimization

**NUMA-Aware Programming** is **the practice of structuring parallel applications to account for Non-Uniform Memory Access architecture — where memory access latency and bandwidth depend on the physical distance between the processor core and the memory controller, with local access being 1.5-3× faster than remote access across interconnect links**. **NUMA Architecture:** - **NUMA Nodes**: each processor socket (or chiplet cluster) has a local memory controller and attached DRAM — accessing local memory takes ~80 ns while remote memory access through interconnect (QPI, UPI, Infinity Fabric) takes ~120-250 ns - **Topology Discovery**: operating systems expose NUMA topology through sysfs (/sys/devices/system/node/) or hwloc library — applications query topology to determine which cores belong to which NUMA nodes and the distance matrix between nodes - **Interconnect Bandwidth**: inter-socket links provide 50-200 GB/s depending on generation — saturating remote bandwidth with memory-intensive workloads causes severe contention and performance degradation - **Multi-Socket Servers**: 2-socket and 4-socket servers are common in HPC and enterprise — 4-socket systems have 2-hop remote access adding additional latency; 8-socket systems (rare) have even deeper NUMA hierarchies **Memory Allocation Policies:** - **First-Touch Policy**: default Linux policy — memory pages allocated on the NUMA node where the first accessing thread runs; initialization pattern determines permanent placement - **Interleave Policy**: pages round-robin across all NUMA nodes — provides average performance across all cores but optimal for no specific core; useful for shared data accessed equally by all threads - **NUMA-Bind Policy**: explicitly bind allocation to a specific node — ensures data stays local to the threads that access it; implemented via numactl --membind or numa_alloc_onnode() - **Migration**: transparent page migration moves pages closer to their most frequent accessor — enabled via AutoNUMA/NUMA balancing in Linux kernel; adds overhead but automatically corrects poor initial placement **Thread Affinity and Binding:** - **Thread Pinning**: bind threads to specific cores using pthread_setaffinity or OMP_PROC_BIND — prevents migration that would separate a thread from its local memory, catastrophically increasing access latency - **Core Binding Strategies**: close binding (fill one socket first) maximizes cache sharing; spread binding (distribute across sockets) maximizes total bandwidth — optimal strategy depends on workload characteristics - **Hyper-Threading Considerations**: binding compute-intensive threads to physical cores (not HT siblings) avoids resource contention — memory-intensive threads may benefit from HT by overlapping computation with memory stalls **NUMA-aware programming is essential for achieving scalable performance on modern multi-socket servers — applications that ignore NUMA topology commonly lose 30-50% of theoretical performance due to remote memory access penalties and interconnect contention.**