GPU Interconnect Technologies (NVLink vs. PCIe vs. NVSwitch) are the communication fabrics that connect GPUs to each other and to CPUs — where the bandwidth, latency, and topology of these interconnects critically determine multi-GPU training performance, as gradient synchronization and tensor parallelism require moving terabytes of data between GPUs per second, making interconnect choice the primary bottleneck differentiator between consumer and data center GPU systems.

Interconnect Comparison

NVLink Architecture

• NVLink is NVIDIA's proprietary high-speed GPU-to-GPU interconnect.
• Each NVLink lane: 25 GB/s (NVLink 3) → 50 GB/s (NVLink 4) → 100 GB/s (NVLink 5).
• H100: 18 NVLink 4 lanes = 900 GB/s bidirectional → 14× PCIe 5.0 bandwidth.
• Direct GPU-to-GPU memory access: GPU 0 can read/write GPU 1 memory at full NVLink speed.

NVSwitch

• NVSwitch: Dedicated switch chip that connects multiple GPUs via NVLink.
• DGX H100: 4 NVSwitch chips connect 8 H100 GPUs → any-to-any full bandwidth.
• Without NVSwitch: Only nearest-neighbor NVLink connections → limited topology.
• With NVSwitch: Full bisection bandwidth → AllReduce at full speed regardless of communication pattern.

Multi-Node: NVLink + InfiniBand

<svg viewBox="0 0 578 150" xmlns="http://www.w3.org/2000/svg" style="max-width:100%;height:auto" role="img"><rect x="0" y="0" width="578" height="150" rx="12" fill="#0d1117"/><g font-family="ui-monospace,SFMono-Regular,Menlo,Consolas,&quot;Liberation Mono&quot;,monospace" font-size="14"><text xml:space="preserve" x="20" y="31.7"><tspan fill="#c9d1d9"> Node 0:                          Node 1:</tspan></text><text xml:space="preserve" x="20" y="50.7"><tspan fill="#c9d1d9"> [GPU0]</tspan><tspan fill="#6e7681">──</tspan><tspan fill="#c9d1d9">NVLink</tspan><tspan fill="#6e7681">──</tspan><tspan fill="#c9d1d9">[GPU1]          [GPU4]</tspan><tspan fill="#6e7681">──</tspan><tspan fill="#c9d1d9">NVLink</tspan><tspan fill="#6e7681">──</tspan><tspan fill="#c9d1d9">[GPU5]</tspan></text><text xml:space="preserve" x="20" y="69.7"><tspan fill="#c9d1d9"> [GPU2]</tspan><tspan fill="#6e7681">──</tspan><tspan fill="#c9d1d9">NVLink</tspan><tspan fill="#6e7681">──</tspan><tspan fill="#c9d1d9">[GPU3]          [GPU6]</tspan><tspan fill="#6e7681">──</tspan><tspan fill="#c9d1d9">NVLink</tspan><tspan fill="#6e7681">──</tspan><tspan fill="#c9d1d9">[GPU7]</tspan></text><text xml:space="preserve" x="20" y="88.7"><tspan fill="#c9d1d9">    All connected via NVSwitch        All connected via NVSwitch</tspan></text><text xml:space="preserve" x="20" y="107.7"><tspan fill="#c9d1d9">         |                                  |</tspan></text><text xml:space="preserve" x="20" y="126.7"><tspan fill="#c9d1d9">    InfiniBand 400G </tspan><tspan fill="#6e7681">────────────</tspan><tspan fill="#c9d1d9"> InfiniBand 400G</tspan></text></g></svg>

• Intra-node: NVLink (900 GB/s) → fast tensor/pipeline parallelism.
• Inter-node: InfiniBand (50-100 GB/s) → data parallelism gradient sync.
• Hierarchy: Optimize communication to keep most traffic intra-node.

Impact on ML Training

PCIe Still Matters

• CPU-GPU data transfer (dataset loading): PCIe 5.0 is sufficient.
• Consumer GPUs: NVLink not available → PCIe only.
• Inference serving: PCIe bandwidth often sufficient for batch inference.
• Cost: PCIe switches are commodity; NVSwitch is expensive and NVIDIA-exclusive.

GPU interconnect technology is the infrastructure that makes large-scale AI training possible — the 10-30× bandwidth advantage of NVLink over PCIe is what enables tensor parallelism across GPUs, without which training models larger than single-GPU memory would require prohibitively slow PCIe communication, and the NVSwitch full-mesh topology is what makes 8-GPU DGX systems behave like a single massive accelerator.