nvlink nvswitch

**NVLink and NVSwitch** are **NVIDIA's proprietary high-bandwidth, low-latency interconnect technologies that connect GPUs within a server at bandwidths far exceeding PCIe — where NVLink provides point-to-point GPU-to-GPU connections at 900 GB/s bidirectional (H100) and NVSwitch creates a fully-connected all-to-all fabric among 8 GPUs, enabling the GPU-to-GPU communication bandwidth required for efficient tensor and data parallelism in large-scale AI training**. **Why PCIe Is Insufficient** PCIe 5.0 x16 provides 64 GB/s bidirectional bandwidth. An H100 GPU generates 3.35 PFLOPS of compute and has 3.35 TB/s of HBM bandwidth. If inter-GPU communication is limited to 64 GB/s, the GPU spends >90% of distributed training time waiting for data transfers. NVLink provides 900 GB/s — 14x PCIe — making inter-GPU communication nearly as fast as local memory access. **NVLink Architecture** NVLink consists of high-speed serial links using proprietary signaling: - **NVLink 4.0 (H100)**: 18 links per GPU, each 25 GB/s per direction → 450 GB/s per direction, 900 GB/s bidirectional total. - **NVLink 5.0 (B200)**: 18 links at 50 GB/s each → 900 GB/s per direction, 1.8 TB/s bidirectional. Each link is a direct, dedicated connection — not shared bus. Multiple links can connect the same GPU pair for higher bandwidth, or spread across multiple GPU pairs for connectivity. **NVSwitch: All-to-All Fabric** Connecting 8 GPUs with point-to-point NVLink requires each GPU to dedicate links to 7 others — consuming all available links. NVSwitch is a dedicated crossbar switch chip that aggregates NVLink connections: - Each GPU connects all its NVLink lanes to NVSwitch chips. - NVSwitch routes any-to-any GPU traffic through the switch fabric. - DGX H100: 4 NVSwitch chips provide full bisection bandwidth — any GPU can communicate with any other GPU at full 900 GB/s simultaneously. **Multi-Node Scaling (NVLink Network)** DGX SuperPOD and GB200 NVL72 extend the NVSwitch fabric across multiple nodes: - GB200 NVL72: 72 GPUs connected through a 5th-generation NVSwitch fabric as a single, flat NVLink domain. Every GPU can access every other GPU's memory at NVLink speed — no PCIe or InfiniBand bottleneck within the domain. - For larger clusters: NVLink domains are connected via InfiniBand NDR (400 Gbps), creating a two-tier network (fast intra-domain, slower inter-domain). **Software Integration** NCCL (NVIDIA Collective Communications Library) automatically detects the NVLink/NVSwitch topology and maps collective operations (allreduce, allgather) to optimal ring or tree patterns over the physical links. CUDA-aware MPI implementations use NVLink for intra-node communication and InfiniBand for inter-node. NVLink and NVSwitch are **the private highway system that NVIDIA built because the public roads (PCIe) could not handle GPU traffic** — enabling multi-GPU systems to operate as a unified compute engine rather than a collection of loosely-connected accelerators.