NUMA Architecture and Memory Affinity enable explicit placement of data and threads on multi-socket systems to exploit local memory bandwidth and latency, critical for HPC and data-center applications scaling to 100s of cores.

Non-Uniform Memory Access Topology

• NUMA Organization: Multiple sockets (CPUs), each with local memory attached. Local socket memory ~100ns latency, remote socket memory ~200-400ns (2-4x penalty).
• Memory Bandwidth Asymmetry: Local DRAM bandwidth (say 100 GB/s) shared with other local cores. Remote DRAM bandwidth crossed via QPI/Infinity Fabric interconnect (less bandwidth than local).
• Example Topology: Dual-socket Xeon with 32 cores per socket. Each core can access both socket's memory, but local access preferred.
• UMA vs NUMA: Older systems uniform memory access (UMA) via shared front-side bus. Modern systems inherently NUMA due to scaling limitations of centralized memory controller.

NUMA Node Binding and Thread Affinity

• NUMA Node Definition: Logical grouping of cores + associated memory. Socket-based binding: threads pinned to cores in same socket as their data.
• numactl Command: numactl --membind=node0 --cpunodebind=node0 application. Forces threads/memory to specific NUMA node. Prevents OS migration.
• libnuma Library: Programmatic NUMA control. numa_alloc_onnode(), numa_bind(), numa_set_preferred(). Enables application-level NUMA awareness.
• cpuset Cgroups: Linux control groups restrict processes to CPU/memory subsets. System-wide NUMA orchestration via cgroups.

First-Touch Policy

• Memory Allocation Mechanism: Pages allocated to NUMA node of thread first accessing page (write). OS tracks page residency.
• Default Behavior: malloc() allocates from kernel's allocator, typically interleaved across nodes (round-robin). Application overrides via numa_alloc_onnode().
• First-Touch Implication: Thread A allocates array B but doesn't initialize; Thread B initializes B. B ends up on B's node (correct affinity).
• Guideline: Initialize data on thread that will access it, or explicitly allocate on target node before other threads touch.

Remote vs Local Memory Latency Impact

• Latency Difference: Local ~100ns, remote ~300ns (3x penalty). Impacts iterative workloads (large loop counts × remote access = significant slowdown).
• Bandwidth Scaling: Remote bandwidth congested by all-to-all access patterns. Single-socket bandwidth ~100 GB/s; multi-socket aggregate ~150-200 GB/s (sub-linear).
• Cache Effects: L3 cache (8-20 MB per socket) mitigates some remote access penalties. If working set fits in L3, remote penalty minimal.
• Example Impact: 1000-iteration loop accessing remote memory: 1000 × 200ns = 200µs (remote) vs 100µs (local). 2x slowdown possible.

NUMA-Aware Data Structures

• Replicated Data: Hot data replicated per socket (each socket has copy). Slight memory overhead but eliminates remote access.
• Data Partitioning: Divide large arrays by NUMA node. Thread i processes array[i×partition_size:(i+1)×partition_size]. Guarantees local access.
• Hash Table Striping: Hash table buckets assigned to NUMA nodes. Hash function distributes keys across nodes balancing load and access locality.
• Graph Partitioning: Graph algorithms (matrix computations, machine learning) partition vertices/edges by NUMA locality. Minimize cross-node edges.

Memory Interleaving vs Binding

• Interleaved Mode: OS spreads pages round-robin across NUMA nodes. Balances memory usage but serializes remote access across all nodes. Poor latency.
• Bound Mode: Pages allocated on specific node. Requires explicit NUMA awareness (application or numactl). Excellent latency but requires work distribution matching binding.
• Hybrid Approaches: Bind hot/critical data to local node, interleave cold data. Best of both worlds.

NUMA Scheduling and OS Coordination

• OS NUMA Scheduler: Linux kernel scheduler (CFS) considers NUMA locality. Migrates threads toward memory (if cheaper than migrating memory).
• Task Scheduler Trade-offs: Migrate thread (cache cold) vs keep thread (remote memory). Decision based on current load, task runtime, memory intensity.
• AutoNUMA: Linux feature periodically migrates pages toward threads that access them (and vice versa). Reduces manual tuning but adds overhead.

NUMA in Multi-Socket HPC Servers

• Dual/Quad Socket Systems: 2-4 sockets per server, 64-256 cores total. Typical HPC configuration in data centers.
• Binding Strategy: MPI ranks bound to NUMA nodes (one rank per node). Inter-rank communication via network (InfiniBand) not NUMA crossings.
• Memory Scaling: Dual-socket Xeon: 256 GB-1 TB memory (128GB-512GB per socket). Single-node jobs fit; larger jobs spill to other nodes (network-based, slower).
• Benchmark Sensitivity: STREAM benchmark 5-10x slower on remote nodes vs local. Gemm (compute-bound) largely unaffected by NUMA.