peer to peer gpu
**Peer-to-Peer (P2P) GPU Communication** is the **hardware and software capability that allows one GPU to directly access another GPU's memory without routing data through CPU main memory** — eliminating the host memory copy bottleneck in multi-GPU systems, reducing transfer latency by 2-5×, and enabling programming models where GPUs transparently share data, essential for multi-GPU deep learning training, scientific simulation, and real-time rendering.
**P2P Communication Paths**
| Path | How | Bandwidth | Latency |
|------|-----|-----------|--------|
| Traditional (staged) | GPU A → CPU RAM → GPU B | Limited by PCIe + memcpy | ~10-20 µs |
| P2P over PCIe | GPU A → PCIe switch → GPU B | PCIe speed (32-64 GB/s) | ~3-5 µs |
| P2P over NVLink | GPU A → NVLink → GPU B | NVLink speed (600-900 GB/s) | ~1-2 µs |
| GPUDirect RDMA | Network → GPU (bypass CPU) | Network speed (25-100 GB/s) | ~2-5 µs |
**CUDA P2P API**
```cuda
// Check P2P support
int canAccess;
cudaDeviceCanAccessPeer(&canAccess, gpu0, gpu1);
// Enable P2P access
cudaSetDevice(gpu0);
cudaDeviceEnablePeerAccess(gpu1, 0);
// Direct copy between GPUs (no CPU staging)
cudaMemcpyPeer(dst_ptr_gpu1, gpu1, src_ptr_gpu0, gpu0, size);
// Async P2P copy
cudaMemcpyPeerAsync(dst, gpu1, src, gpu0, size, stream);
// Direct pointer access (Unified Virtual Addressing)
// GPU 0 kernel can dereference pointer to GPU 1 memory
my_kernel<<>>(gpu1_ptr); // Access remote GPU memory
```
**GPUDirect Technologies (NVIDIA)**
| Technology | What | Bypass |
|-----------|------|--------|
| GPUDirect P2P | GPU-to-GPU over PCIe/NVLink | CPU memory |
| GPUDirect RDMA | Network NIC → GPU directly | CPU memory + CPU |
| GPUDirect Storage | NVMe SSD → GPU directly | CPU memory + filesystem |
| GPUDirect Async | Async control of all above | CPU involvement |
**GPUDirect RDMA (Network → GPU)**
- InfiniBand NIC reads/writes GPU memory directly.
- MPI_Send from GPU → NIC grabs data directly from GPU memory → sends over network → remote NIC writes directly to remote GPU.
- No CPU copies in the data path → critical for distributed training.
- Requires: NVIDIA GPU + Mellanox/NVIDIA NIC + GPUDirect-aware driver.
**P2P Topology Awareness**
```
GPU 0 ←NVLink→ GPU 1
↑ ↑
NVLink NVLink
↓ ↓
GPU 2 ←NVLink→ GPU 3
↑ ↑
PCIe PCIe
↓ ↓
CPU Socket 0 CPU Socket 1
```
- GPU 0 → GPU 1 (NVLink): ~600 GB/s, ~1 µs.
- GPU 0 → GPU 3 (NVLink via switch): ~600 GB/s, ~1.5 µs.
- GPU 0 → GPU on remote socket (PCIe): ~25 GB/s, ~5 µs.
- Training frameworks (PyTorch, DeepSpeed) should be topology-aware → minimize cross-socket transfers.
**Impact on ML Training**
- AllReduce: P2P NVLink → ring topology at 600 GB/s → fast gradient sync.
- Tensor parallelism: Each GPU holds fraction of layer → P2P required for activations.
- Expert parallelism (MoE): Tokens routed to expert GPUs → P2P for token transfer.
- Without P2P: All traffic goes through CPU → 10× slower → multi-GPU training impractical.
Peer-to-peer GPU communication is **the physical foundation of multi-GPU computing** — by enabling GPUs to share data at NVLink or PCIe speeds without CPU intermediation, P2P transforms a collection of discrete GPUs into a unified computational fabric where tensor and pipeline parallelism can operate at the bandwidth required by modern large-scale AI training.