numa aware programming

**NUMA-Aware Programming** is the **performance optimization discipline for multi-socket and chiplet-based systems where memory access latency and bandwidth depend on the physical location of the memory relative to the processor — where NUMA-oblivious code can suffer 2-4x performance degradation because remote memory accesses (cross-socket or cross-chiplet) take 1.5-3x longer than local accesses, making data placement and thread affinity the dominant factors in memory-bound application performance**. **NUMA Architecture** In a NUMA system, each processor (socket/chiplet) has its own local memory controller and DRAM. Accessing local memory: ~80-100 ns. Accessing remote memory (through the interconnect — Intel UPI, AMD Infinity Fabric): ~130-200 ns. The latency asymmetry is the "non-uniform" in NUMA. **Example: 2-Socket AMD EPYC** Each socket has 4 CCDs (chiplet core dies), each with its own L3 cache and a local slice of the memory channels. Memory access hierarchy: 1. Same CCD L3: ~10 ns 2. Same socket, different CCD: ~30-50 ns 3. Same socket, different memory controller: ~80-100 ns 4. Remote socket: ~130-200 ns **NUMA Optimization Techniques** - **First-Touch Allocation**: Linux NUMA default policy. Memory pages are allocated on the NUMA node of the first thread that touches (writes to) them. If the initializing thread is on node 0 but the computing thread is on node 1, all accesses are remote. Fix: initialize data on the same threads that will process it. - **Thread-Memory Affinity**: Bind threads to specific cores/NUMA nodes using `numactl --cpunodebind=0 --membind=0`, `sched_setaffinity()`, or OpenMP `OMP_PLACES=cores OMP_PROC_BIND=close`. Ensures threads access local memory. - **Interleaved Allocation**: `numactl --interleave=all` distributes pages round-robin across all nodes. Provides uniform average latency at the cost of no locality optimization. Useful for shared data accessed by all nodes equally. - **NUMA-Aware Data Structures**: Allocate per-node copies of frequently-read data (replication). For producer-consumer patterns, place the buffer on the consumer's node (reads are more latency-sensitive than writes due to store buffers). **Detecting NUMA Issues** - `numastat -p `: Shows per-node memory allocation and remote access counts. - `perf stat -e node-load-misses,node-store-misses`: Hardware counters for remote memory accesses. - Intel VTune / AMD uProf: NUMA-specific analysis modes visualize memory access locality. **NUMA in Practice** - **Databases**: PostgreSQL, MySQL allocate buffer pools NUMA-aware. Connection threads are pinned to the same node as their buffer pages. - **HPC**: MPI rank placement matches NUMA topology. One rank per NUMA node, with OpenMP threads within each rank placed on the same node. - **Cloud/VMs**: VM placement must respect NUMA boundaries. A VM spanning two NUMA nodes suffers remote access penalties on half its memory. **NUMA-Aware Programming is the essential optimization for modern multi-socket and chiplet servers** — ensuring that data lives close to the processor that uses it, because in a NUMA system, WHERE you allocate memory matters as much as HOW you access it.