peer-to-peer gpu communication

**Peer-to-peer GPU communication** is the **direct data transfer between GPUs without staging through host memory** - it lowers latency and improves bandwidth for multi-GPU workloads with frequent inter-device exchange. **What Is Peer-to-peer GPU communication?** - **Definition**: GPU-to-GPU memory copy or access over NVLink or PCIe peer paths. - **Bypass Advantage**: Avoids two-hop host staging that adds copy overhead and CPU involvement. - **Topology Dependence**: Performance depends on whether GPUs share direct links and switch paths. - **Workload Context**: Critical for model parallel and collective communication-heavy training. **Why Peer-to-peer GPU communication Matters** - **Latency Reduction**: Direct paths shorten transfer time for synchronization and activation exchange. - **Bandwidth Gains**: Peer links often provide higher throughput than host-mediated transfer routes. - **CPU Offload**: Less host involvement frees CPU resources for orchestration and data prep. - **Scale Performance**: Efficient P2P is essential for high utilization in dense multi-GPU nodes. - **Communication Overlap**: Faster transfer paths improve potential for compute-communication concurrency. **How It Is Used in Practice** - **Topology Mapping**: Place communication-heavy ranks on GPUs with strongest peer connectivity. - **Capability Checks**: Enable and verify peer access support in runtime initialization. - **Transfer Profiling**: Benchmark peer bandwidth and latency to validate expected path efficiency. Peer-to-peer GPU communication is **a key enabler of efficient multi-GPU execution** - direct device links remove host bottlenecks and improve distributed training throughput.