NVLink is NVIDIA's high-bandwidth interconnect for GPU-to-GPU and GPU-to-CPU communication — providing 600-900 GB/s bidirectional bandwidth compared to PCIe's 64 GB/s, enabling efficient multi-GPU scaling for large model training and inference.
What Is NVLink?
- Definition: Proprietary high-speed GPU interconnect.
- Purpose: Fast multi-GPU communication.
- Bandwidth: 10-14× faster than PCIe Gen5.
- Use Cases: Multi-GPU training, large model sharding.
Why NVLink Matters
- Model Parallelism: Large models span multiple GPUs.
- Gradient Sync: Training requires fast parameter updates.
- Memory Pooling: Access memory across GPUs.
- Inference: Large models need GPU sharding.
- Scaling Efficiency: Minimizes communication bottleneck.
Bandwidth Comparison
Interconnect Speeds:
Interconnect | Bandwidth (Bi-dir) | Generation
------------------|-------------------|------------
NVLink 4 (Hopper) | 900 GB/s | H100
NVLink 3 (Ampere) | 600 GB/s | A100
NVLink 2 (Volta) | 300 GB/s | V100
PCIe Gen5 | 64 GB/s (×16) | Current
PCIe Gen4 | 32 GB/s (×16) | Previous
InfiniBand NDR | 400 Gbps per port | Network
Practical Impact:
Operation | PCIe Gen5 | NVLink 4
-----------------------|--------------|----------
Copy 80GB (A100 mem) | 1.25 sec | 0.13 sec
Gradient sync (10GB) | 156 ms | 11 ms
AllReduce efficiency | 70-80% | 95%+
NVLink Topologies
DGX H100 Topology:
<svg viewBox="0 0 760 470" xmlns="http://www.w3.org/2000/svg" font-family="-apple-system,Segoe UI,Roboto,sans-serif">
<rect x="0" y="0" width="760" height="470" fill="#0d1117"/>
<text x="380" y="28" fill="#e6edf3" font-size="21" font-weight="700" text-anchor="middle">NVLink + NVSwitch — GPU-to-GPU Interconnect</text>
<text x="380" y="48" fill="#8b98a5" font-size="12" text-anchor="middle">900 GB/s per GPU bidirectional — every GPU talks to every other at full bandwidth through NVSwitch</text>
<!-- === LEFT: 8-GPU DGX topology (the money diagram) === -->
<rect x="30" y="62" width="380" height="300" rx="6" fill="#080d14" stroke="#233043" stroke-width="1.2"/>
<text x="220" y="82" fill="#e6edf3" font-size="11" text-anchor="middle" font-weight="600">DGX H100 — 8 GPUs via NVSwitch (all-to-all)</text>
<!-- 8 GPUs arranged in a ring/grid -->
<!-- Top row: GPU 0-3 -->
<rect x="70" y="100" width="55" height="36" rx="4" fill="#122040" stroke="#76b900" stroke-width="1.3"/>
<text x="97" y="118" fill="#76b900" font-size="9" text-anchor="middle" font-weight="600">GPU 0</text>
<text x="97" y="130" fill="#6b7684" font-size="7" text-anchor="middle">H100</text>
<rect x="145" y="100" width="55" height="36" rx="4" fill="#122040" stroke="#76b900" stroke-width="1.3"/>
<text x="172" y="118" fill="#76b900" font-size="9" text-anchor="middle" font-weight="600">GPU 1</text>
<text x="172" y="130" fill="#6b7684" font-size="7" text-anchor="middle">H100</text>
<rect x="220" y="100" width="55" height="36" rx="4" fill="#122040" stroke="#76b900" stroke-width="1.3"/>
<text x="247" y="118" fill="#76b900" font-size="9" text-anchor="middle" font-weight="600">GPU 2</text>
<text x="247" y="130" fill="#6b7684" font-size="7" text-anchor="middle">H100</text>
<rect x="295" y="100" width="55" height="36" rx="4" fill="#122040" stroke="#76b900" stroke-width="1.3"/>
<text x="322" y="118" fill="#76b900" font-size="9" text-anchor="middle" font-weight="600">GPU 3</text>
<text x="322" y="130" fill="#6b7684" font-size="7" text-anchor="middle">H100</text>
<!-- Bottom row: GPU 4-7 -->
<rect x="70" y="250" width="55" height="36" rx="4" fill="#122040" stroke="#76b900" stroke-width="1.3"/>
<text x="97" y="268" fill="#76b900" font-size="9" text-anchor="middle" font-weight="600">GPU 4</text>
<rect x="145" y="250" width="55" height="36" rx="4" fill="#122040" stroke="#76b900" stroke-width="1.3"/>
<text x="172" y="268" fill="#76b900" font-size="9" text-anchor="middle" font-weight="600">GPU 5</text>
<rect x="220" y="250" width="55" height="36" rx="4" fill="#122040" stroke="#76b900" stroke-width="1.3"/>
<text x="247" y="268" fill="#76b900" font-size="9" text-anchor="middle" font-weight="600">GPU 6</text>
<rect x="295" y="250" width="55" height="36" rx="4" fill="#122040" stroke="#76b900" stroke-width="1.3"/>
<text x="322" y="268" fill="#76b900" font-size="9" text-anchor="middle" font-weight="600">GPU 7</text>
<!-- NVSwitch chips (4 in the middle) -->
<rect x="100" y="168" width="50" height="24" rx="3" fill="#2a1a0a" stroke="#f59e0b" stroke-width="1.2"/>
<text x="125" y="184" fill="#fbbf24" font-size="8" text-anchor="middle">NVSw</text>
<rect x="165" y="168" width="50" height="24" rx="3" fill="#2a1a0a" stroke="#f59e0b" stroke-width="1.2"/>
<text x="190" y="184" fill="#fbbf24" font-size="8" text-anchor="middle">NVSw</text>
<rect x="230" y="168" width="50" height="24" rx="3" fill="#2a1a0a" stroke="#f59e0b" stroke-width="1.2"/>
<text x="255" y="184" fill="#fbbf24" font-size="8" text-anchor="middle">NVSw</text>
<rect x="295" y="168" width="50" height="24" rx="3" fill="#2a1a0a" stroke="#f59e0b" stroke-width="1.2"/>
<text x="320" y="184" fill="#fbbf24" font-size="8" text-anchor="middle">NVSw</text>
<!-- NVLink connections (top GPUs to switches) -->
<g stroke="#76b900" stroke-width="0.8" opacity="0.5">
<!-- GPU 0 to all switches -->
<line x1="97" y1="136" x2="125" y2="168"/><line x1="97" y1="136" x2="190" y2="168"/>
<line x1="97" y1="136" x2="255" y2="168"/><line x1="97" y1="136" x2="320" y2="168"/>
<!-- GPU 1 -->
<line x1="172" y1="136" x2="125" y2="168"/><line x1="172" y1="136" x2="190" y2="168"/>
<line x1="172" y1="136" x2="255" y2="168"/><line x1="172" y1="136" x2="320" y2="168"/>
<!-- GPU 2 -->
<line x1="247" y1="136" x2="125" y2="168"/><line x1="247" y1="136" x2="190" y2="168"/>
<line x1="247" y1="136" x2="255" y2="168"/><line x1="247" y1="136" x2="320" y2="168"/>
<!-- GPU 3 -->
<line x1="322" y1="136" x2="125" y2="168"/><line x1="322" y1="136" x2="190" y2="168"/>
<line x1="322" y1="136" x2="255" y2="168"/><line x1="322" y1="136" x2="320" y2="168"/>
<!-- GPU 4-7 to switches (bottom) -->
<line x1="97" y1="250" x2="125" y2="192"/><line x1="97" y1="250" x2="190" y2="192"/>
<line x1="172" y1="250" x2="190" y2="192"/><line x1="172" y1="250" x2="255" y2="192"/>
<line x1="247" y1="250" x2="255" y2="192"/><line x1="247" y1="250" x2="320" y2="192"/>
<line x1="322" y1="250" x2="320" y2="192"/><line x1="322" y1="250" x2="125" y2="192"/>
</g>
<!-- Bandwidth annotation -->
<text x="220" y="210" fill="#76b900" font-size="9" text-anchor="middle" font-weight="600">all-to-all: 900 GB/s per GPU</text>
<text x="220" y="225" fill="#8b98a5" font-size="8" text-anchor="middle">18 NVLink lanes per GPU × 50 GB/s each</text>
<!-- Total bandwidth -->
<text x="220" y="305" fill="#8b98a5" font-size="9" text-anchor="middle">Total bisection BW: 3.6 TB/s (8 GPU node)</text>
<text x="220" y="320" fill="#6b7684" font-size="8.5" text-anchor="middle">vs PCIe Gen5: 64 GB/s per GPU (14x less)</text>
<!-- PCIe connection to CPU (below) -->
<rect x="155" y="332" width="110" height="20" rx="3" fill="#14202c" stroke="#6b7684" stroke-width="0.8"/>
<text x="210" y="346" fill="#8b98a5" font-size="8" text-anchor="middle">CPU (PCIe Gen5 x16)</text>
<line x1="210" y1="286" x2="210" y2="332" stroke="#6b7684" stroke-width="0.5" stroke-dasharray="3,2"/>
<text x="225" y="315" fill="#6b7684" font-size="7">host access</text>
<!-- === RIGHT: NVLink evolution + scale-out === -->
<rect x="430" y="62" width="300" height="120" rx="6" fill="#0b1220" stroke="#233043" stroke-width="1"/>
<text x="580" y="80" fill="#e6edf3" font-size="11" text-anchor="middle" font-weight="600">NVLink Generations</text>
<text x="450" y="100" fill="#6b7684" font-size="9">NVLink 1 (P100): 160 GB/s, 4 links</text>
<text x="450" y="116" fill="#8b98a5" font-size="9">NVLink 3 (A100): 600 GB/s, 12 links</text>
<text x="450" y="132" fill="#76b900" font-size="9" font-weight="600">NVLink 4 (H100): 900 GB/s, 18 links</text>
<text x="450" y="148" fill="#fbbf24" font-size="9" font-weight="600">NVLink 5 (B200): 1.8 TB/s, 18 links</text>
<text x="450" y="168" fill="#6b7684" font-size="8">each gen: ~2x BW through wider lanes + faster signaling</text>
<!-- Scale-out: multi-node -->
<rect x="430" y="190" width="300" height="100" rx="6" fill="#0b1220" stroke="#233043" stroke-width="1"/>
<text x="580" y="208" fill="#e6edf3" font-size="11" text-anchor="middle" font-weight="600">Scale-Out (Multi-Node)</text>
<text x="450" y="228" fill="#76b900" font-size="9.5">Intra-node: NVLink (900 GB/s)</text>
<text x="450" y="244" fill="#60a5fa" font-size="9.5">Inter-node: InfiniBand 400G (50 GB/s)</text>
<text x="450" y="260" fill="#c4b5fd" font-size="9.5">Future: NVLink-Network (72 GPUs, no IB)</text>
<text x="450" y="278" fill="#6b7684" font-size="8.5">Blackwell NVL72: 72 GPUs in one NVLink domain</text>
<text x="450" y="290" fill="#6b7684" font-size="8">= 130 TB/s aggregate — a single logical GPU</text>
<!-- Why it matters -->
<rect x="430" y="298" width="300" height="65" rx="6" fill="#0b1220" stroke="#233043" stroke-width="1"/>
<text x="580" y="316" fill="#e6edf3" font-size="10" text-anchor="middle" font-weight="600">Why NVLink Matters for AI Training</text>
<text x="450" y="336" fill="#8b98a5" font-size="9">AllReduce (gradient sync) dominates at scale</text>
<text x="450" y="352" fill="#8b98a5" font-size="9">Tensor parallel needs fast GPU↔GPU</text>
<text x="450" y="366" fill="#6b7684" font-size="8.5">BW-limited: time_comm = params / BW per step</text>
<!-- Bottom -->
<text x="380" y="395" fill="#e6edf3" font-size="10" text-anchor="middle" font-weight="600">NVLink is the reason you can't just use PCIe GPUs for AI training — parallelism demands high-BW interconnect</text>
<text x="380" y="412" fill="#8b98a5" font-size="9.5" text-anchor="middle">Physical: SerDes lanes on GPU die edge, differential signaling, NVSwitch is a separate ASIC (TSMC 4N)</text>
<text x="380" y="440" fill="#6b7684" font-size="10" text-anchor="middle">PCIe is a road. NVLink is a highway. At scale, the highway width determines how fast you can train.</text>
<text x="380" y="460" fill="#6b7684" font-size="11" text-anchor="middle">NVLink turns 8 GPUs into one coherent compute pool — the interconnect fabric is as important as the GPU itself.</text>
</svg>
Consumer NVLink (RTX 4090):
3090: NVLink bridge, 2 GPUs
4090: No NVLink support
NVSwitch
What It Enables:
Without NVSwitch:
- Direct links only between neighbor GPUs
- Limited topology
With NVSwitch:
- All-to-all connectivity
- Full bisection bandwidth
- Any GPU reaches any GPU directly
DGX Generations:
System | GPUs | Topology | GPU-GPU BW
-------------|------|---------------------|------------
DGX A100 | 8 | NVSwitch (full) | 600 GB/s
DGX H100 | 8 | NVSwitch (full) | 900 GB/s
DGX GH200 | 256 | Grace Hopper + NVL | 900 GB/s
Programming with NVLink
NCCL (NVIDIA Collective Communications Library):
import torch
import torch.distributed as dist
# Initialize with NCCL backend (uses NVLink automatically)
dist.init_process_group(backend="nccl")
# AllReduce uses NVLink when available
tensor = torch.randn(1000, device="cuda")
dist.all_reduce(tensor) # Automatically uses NVLink
Peer-to-Peer Memory Access:
// Enable P2P access between GPUs
cudaDeviceEnablePeerAccess(peer_device, 0);
// Direct memory access across NVLink
cudaMemcpyPeer(dst, dstDevice, src, srcDevice, size);
Checking NVLink:
# Check NVLink status
nvidia-smi nvlink -s
# Show topology
nvidia-smi topo -m
# NVLink utilization
nvidia-smi nvlink -g 0
NVLink vs. PCIe Use Cases
Use Case | Best Interconnect
----------------------|------------------
Single GPU inference | PCIe (sufficient)
Multi-GPU training | NVLink (essential)
Large model inference | NVLink (model sharding)
Consumer workstation | PCIe (NVLink limited)
Data center | NVLink + InfiniBand
NVLink is essential infrastructure for multi-GPU AI — without high-bandwidth interconnects, scaling to multiple GPUs becomes inefficient as communication overhead dominates, making NVLink critical for training large models and serving them across GPU clusters.
nvlinkpcieinterconnectbandwidthgpunvswitchnccl
Explore 500+ Semiconductor & AI Topics
From EUV lithography to CUDA optimization — search the full knowledge base or chat with our AI assistant.