numa architecture

**NUMA Architecture and Optimization** is the **multi-processor memory architecture where each processor socket has locally attached memory that it can access faster (50-100 ns) than remote memory attached to another socket (100-200 ns) — creating a non-uniform memory access pattern that requires NUMA-aware software design to ensure that threads access local memory wherever possible, because naive memory allocation can cause 30-50% performance degradation when data is consistently fetched from remote NUMA nodes**. **NUMA Hardware Structure** A 2-socket server with 64 cores per socket: - **NUMA Node 0**: 64 CPU cores + 256 GB local DDR5 (connected directly via integrated memory controller). Local access latency: ~80 ns. - **NUMA Node 1**: 64 CPU cores + 256 GB local DDR5. Local access latency: ~80 ns. - **Interconnect**: UPI (Ultra Path Interconnect, Intel) or Infinity Fabric (AMD) connecting the two sockets. Remote access latency: ~140-180 ns (1.8-2.2x local). **NUMA Ratio**: Remote/Local latency ratio. Typical: 1.5-2.5x. Higher ratios demand more aggressive NUMA optimization. AMD EPYC's chiplet architecture creates multiple NUMA domains (NPS — NUMA Nodes Per Socket) within a single socket. **Memory Allocation Policies** Linux NUMA policies (set via numactl, mbind(), set_mempolicy()): - **Local**: Allocate memory on the NUMA node where the allocating thread is running. Default policy for most allocations. - **Bind**: Restrict allocation to specific NUMA node(s). Guarantees locality but risks imbalance if the specified node runs out of memory. - **Interleave**: Round-robin page allocation across all NUMA nodes. Ensures even memory distribution at the cost of 50% remote accesses. Good for shared data accessed equally by all threads. - **Preferred**: Try the specified node first; fall back to others if full. **NUMA-Aware Programming** - **First-Touch Policy**: Pages are allocated on the NUMA node of the first thread that writes to them. Consequence: parallel initialization is critical — initialize data structures from the same threads that will process them. Serial initialization followed by parallel computation causes all data to land on node 0. - **Thread Pinning**: Pin threads to specific cores/sockets using pthread_setaffinity_np() or numactl. Prevents the OS scheduler from migrating a thread to a remote node, away from its data. - **Data Partitioning**: Partition data structures so each NUMA node's threads work on locally-allocated portions. Array processing: thread i processes array[i*N/P..(i+1)*N/P] with those pages allocated on thread i's local node. **NUMA in Practice** - **Database Systems**: Query executors are NUMA-aware, routing queries to the socket that holds the relevant data partition. Buffer pool pages are allocated on the NUMA node of the socket that manages the corresponding tablespace. - **JVM NUMA**: Java garbage collectors (ZGC, Shenandoah) support NUMA-aware heap allocation, placing objects on the allocating thread's local node. - **Virtualization**: Virtual machines should be pinned to a single NUMA node with memory allocated from that node. Cross-NUMA VM placement can cause 40-50% performance loss. NUMA Architecture is **the unavoidable physical reality of multi-socket computing** — where the speed of light and electrical signal propagation create inherent latency asymmetry that software must acknowledge and accommodate, turning memory placement and thread affinity into first-class performance optimization concerns.