NVLink and NVSwitch are NVIDIA's proprietary high-bandwidth, low-latency interconnect technologies that connect GPUs within a server at bandwidths far exceeding PCIe — where NVLink provides point-to-point GPU-to-GPU connections at 900 GB/s bidirectional (H100) and NVSwitch creates a fully-connected all-to-all fabric among 8 GPUs, enabling the GPU-to-GPU communication bandwidth required for efficient tensor and data parallelism in large-scale AI training.

Why PCIe Is Insufficient

PCIe 5.0 x16 provides 64 GB/s bidirectional bandwidth. An H100 GPU generates 3.35 PFLOPS of compute and has 3.35 TB/s of HBM bandwidth. If inter-GPU communication is limited to 64 GB/s, the GPU spends >90% of distributed training time waiting for data transfers. NVLink provides 900 GB/s — 14x PCIe — making inter-GPU communication nearly as fast as local memory access.

NVLink Architecture

NVLink consists of high-speed serial links using proprietary signaling:

• NVLink 4.0 (H100): 18 links per GPU, each 25 GB/s per direction → 450 GB/s per direction, 900 GB/s bidirectional total.
• NVLink 5.0 (B200): 18 links at 50 GB/s each → 900 GB/s per direction, 1.8 TB/s bidirectional.

Each link is a direct, dedicated connection — not shared bus. Multiple links can connect the same GPU pair for higher bandwidth, or spread across multiple GPU pairs for connectivity.

NVSwitch: All-to-All Fabric

Connecting 8 GPUs with point-to-point NVLink requires each GPU to dedicate links to 7 others — consuming all available links. NVSwitch is a dedicated crossbar switch chip that aggregates NVLink connections:

• Each GPU connects all its NVLink lanes to NVSwitch chips.
• NVSwitch routes any-to-any GPU traffic through the switch fabric.
• DGX H100: 4 NVSwitch chips provide full bisection bandwidth — any GPU can communicate with any other GPU at full 900 GB/s simultaneously.

Multi-Node Scaling (NVLink Network)

DGX SuperPOD and GB200 NVL72 extend the NVSwitch fabric across multiple nodes:

• GB200 NVL72: 72 GPUs connected through a 5th-generation NVSwitch fabric as a single, flat NVLink domain. Every GPU can access every other GPU's memory at NVLink speed — no PCIe or InfiniBand bottleneck within the domain.
• For larger clusters: NVLink domains are connected via InfiniBand NDR (400 Gbps), creating a two-tier network (fast intra-domain, slower inter-domain).

Software Integration

NCCL (NVIDIA Collective Communications Library) automatically detects the NVLink/NVSwitch topology and maps collective operations (allreduce, allgather) to optimal ring or tree patterns over the physical links. CUDA-aware MPI implementations use NVLink for intra-node communication and InfiniBand for inter-node.

NVLink and NVSwitch are the private highway system that NVIDIA built because the public roads (PCIe) could not handle GPU traffic — enabling multi-GPU systems to operate as a unified compute engine rather than a collection of loosely-connected accelerators.