hardware performance counter monitoring

**Hardware Performance Monitoring: PMU Access and Analysis — performance counter instrumentation revealing CPU behavior (cache, branch prediction, instruction-level parallelism) guiding optimization** **CPU Performance Counters** - **Cycle Count**: clock cycles elapsed (basic metric, used to normalize other counters) - **Instruction Count**: total instructions executed, IPC = instructions/cycles (>1 indicates parallelism, <1 indicates stalls) - **Cache Misses**: L1/L2/L3 cache misses per 1000 instructions, high misses indicate memory bottleneck - **Branch Mispredictions**: incorrect branch predictions, stall pipeline (15-20 cycle penalty typical) - **Specialized**: floating-point ops, vector operations, SIMD utilization, page faults **Top-Down Microarchitecture Analysis (TMA)** - **Frontend/Backend Stalls**: categorize cycles where CPU stalled (frontend: fetch not available, backend: execution blocked) - **Bad Speculation**: cycles wasted on mispredicted branches or speculative execution - **Retiring**: cycles spent on useful work (committed instructions) - **Implication**: identifies where optimization effort should focus (frontend vs backend vs speculation) **Linux perf Tool** - **perf stat**: measure counters for single run (``perf stat ./program'), output avg/total counts - **perf record**: record counter data during execution (``perf record -e cycles,cache-misses ./program'), generates data.perf - **perf report**: analyze recorded data (``perf report'), flame graph shows hot functions - **CPU Event Selection**: vendor-specific (Intel: UOPS_ISSUED, AMD: DISPATCH0_STALLS), requires knowledge of ISA **PAPI (Performance Application Programming Interface)** - **Portable API**: abstract performance counter names (PAPI_L1_DCM = L1 data cache miss, works on Intel/AMD/ARM) - **C Library**: ``#include ', call PAPI_start_counters(), PAPI_read_counters(), PAPI_stop_counters() - **Preset Events**: pre-defined events (PAPI_FP_OPS floating-point ops), user-friendly vs raw PMU events - **Group Recording**: measure multiple counters simultaneously (hardware limit: typically 4-8 concurrent counters) **Intel VTune Profiler** - **GUI Interface**: graphical analysis (vs CLI perf), intuitive timeline visualization - **Multiple Modes**: sampling (record every N cycles), tracing (record all events), metrics (compute derived metrics) - **Hotspot Analysis**: identifies functions consuming most time, drill-down to lines of code - **System-Wide**: profile entire system (all processes), identify unexpected CPU utilization - **License**: commercial (Intel, part of oneAPI toolkit), free for limited academic use **AMD uProf** - **AMD Equivalent**: similar to Intel VTune, optimized for AMD EPYC/Ryzen - **Features**: instruction-based sampling, memory analysis (cache coherency, interconnect) - **Integration**: Linux perf compatibility (can import perf data) - **Cost**: free for AMD customers **NVIDIA Nsight (GPU Profiling)** - **GPU Performance**: kernel occupancy (how many thread blocks executing), memory throughput (coalescing) - **Warp Divergence**: GPU threads (in same warp) diverge (take different branches), serializes execution - **Memory Analysis**: global memory coalescing (contiguous access efficient), local memory usage - **Timeline**: GPU timeline synchronized with CPU timeline (overall system view) **PMU (Performance Monitoring Unit) Programming** - **Linux Perf Events**: perf_event_open() syscall, configure which counter to measure, attach to process/CPU - **Counter Multiplexing**: hardware limit (N concurrent counters), OS time-multiplexes if more requested - **Ring Buffers**: kernel maintains buffer (overflows discard oldest), user-space reads periodically - **Permissions**: typical users require elevated privileges (sysctl perf_event_paranoid), or system admin grant access **Performance Baseline and Comparison** - **Baseline Measurement**: profile unoptimized code (establish starting point), track improvements over iterations - **A/B Testing**: compare two code variants (perf stat -c program_v1, program_v2), identify faster version - **Statistical Significance**: multiple runs (10+), report mean/stddev, account for variance from system noise **Flame Graphs and Visualization** - **Flame Graph**: horizontal bars represent function call stack (height = stack depth), width = time spent - **Hot Paths**: wide functions indicate hot spots (candidates for optimization) - **Color**: typically hue indicates thread, saturation indicates issue type (stalls, cache misses) - **Tool**: brendangregg/FlameGraph (convert perf output to svg visualization) **Cache Analysis and Optimization** - **L1/L2/L3 Miss Rates**: compute miss/hit ratio per level, guide prefetch/memory layout optimization - **Cache Associativity**: capacity misses (conflict misses) if data patterns don't align with cache structure - **Working Set**: estimate how much memory actively used (vs cold data), if >cache capacity: memory bottleneck - **Prefetch Hints**: software hints (PREFETCH instruction) or hardware prefetchers (predictive) **Branch Prediction and Speculation** - **Misprediction Rate**: percentage of branches mispredicted, target <2-3% (modern predictors ~98%+ accuracy) - **Penalty**: misprediction costs 15-25 cycles (pipeline flush), sum mispredictions: significant performance loss - **Optimization**: reduce branches (loop unrolling, predicated execution), improve prediction (data-dependent branches difficult) **Scaling to Many Cores** - **Per-Core Counters**: all cores generate performance data (N cores = N counter streams) - **Aggregation**: typically average/sum across cores, but per-core analysis useful (load imbalance detection) - **Storage**: sampling rates ~1000 Hz typical (per core), 1000 cores = 1M events/sec (significant I/O) **Online vs Offline Analysis** - **Online**: analyze performance during run (adjust knobs if needed), requires minimal overhead - **Offline**: post-mortem analysis (full data capture), enables detailed study but too late for adjustment - **Hybrid**: profile phase (collect data), optimize phase (modify code), repeat **Future Tools and Emerging Standards** - **OpenTelemetry**: standard for observability (logs, metrics, traces), HPC adoption emerging - **eBPF**: kernel event collection (low overhead), emerging alternative to perf (tools like bcc) - **Machine Learning**: automatic anomaly detection (profiler identifies unexpected behavior, alerts user)