NVLink Interconnect is NVIDIA's proprietary high-bandwidth, low-latency GPU-to-GPU interconnect that provides 10-15× higher bandwidth than PCIe — enabling direct GPU memory access at 900 GB/s bidirectional (NVLink 4.0) and sub-microsecond latency, making tightly-coupled multi-GPU systems practical for model parallelism, large-batch training, and unified memory architectures that treat multiple GPUs as a single coherent memory space.

NVLink Architecture:

• Physical Layer: high-speed serial links using PAM4 (4-level pulse amplitude modulation) signaling at 50 Gb/s per lane (NVLink 3.0) or 100 Gb/s (NVLink 4.0); each NVLink comprises multiple lanes bundled into a bidirectional connection
• Link Configuration: H100 GPUs have 18 NVLink connections, each providing 50 GB/s bidirectional (25 GB/s each direction); total 900 GB/s bidirectional per GPU; A100 has 12 NVLinks at 600 GB/s total; compare to PCIe 5.0 x16 at 128 GB/s bidirectional
• Protocol: cache-coherent protocol supporting load/store semantics; GPUs can directly read/write remote GPU memory using standard CUDA memory operations; hardware handles address translation, routing, and coherency
• Topology Flexibility: NVLinks can connect GPUs in various topologies (ring, mesh, hypercube, fully-connected via NVSwitch); topology determines effective bandwidth between non-adjacent GPUs

<svg viewBox="0 0 760 470" xmlns="http://www.w3.org/2000/svg" font-family="-apple-system,Segoe UI,Roboto,sans-serif">
  <rect x="0" y="0" width="760" height="470" fill="#0d1117"/>
  <text x="380" y="28" fill="#e6edf3" font-size="21" font-weight="700" text-anchor="middle">NVLink — High-Bandwidth GPU Interconnect</text>
  <text x="380" y="48" fill="#8b98a5" font-size="12" text-anchor="middle">900 GB/s bidirectional (4th gen) — 7× faster than PCIe 5.0, enables multi-GPU as one memory space</text>

  <!-- Bandwidth comparison -->
  <rect x="30" y="65" width="700" height="130" rx="6" fill="#080d14" stroke="#233043" stroke-width="1"/>
  <text x="380" y="84" fill="#e6edf3" font-size="10" font-weight="600" text-anchor="middle">Interconnect Bandwidth Comparison</text>

  <!-- Bars -->
  <text x="50" y="110" fill="#6b7684" font-size="8">PCIe 4.0:</text>
  <rect x="145" y="102" width="32" height="12" rx="2" fill="#6b7684"/>
  <text x="184" y="112" fill="#6b7684" font-size="7">32 GB/s</text>

  <text x="50" y="130" fill="#60a5fa" font-size="8">PCIe 5.0:</text>
  <rect x="145" y="122" width="63" height="12" rx="2" fill="#60a5fa"/>
  <text x="215" y="132" fill="#93c5fd" font-size="7">64 GB/s</text>

  <text x="50" y="150" fill="#a78bfa" font-size="8">NVLink 3 (A100):</text>
  <rect x="145" y="142" width="300" height="12" rx="2" fill="#a78bfa"/>
  <text x="452" y="152" fill="#c4b5fd" font-size="7">600 GB/s</text>

  <text x="50" y="170" fill="#34d399" font-size="8" font-weight="600">NVLink 4 (H100):</text>
  <rect x="145" y="162" width="450" height="12" rx="2" fill="#34d399"/>
  <text x="602" y="172" fill="#6ee7b7" font-size="7" font-weight="600">900 GB/s</text>

  <text x="50" y="190" fill="#f59e0b" font-size="8">NVLink 5 (B200):</text>
  <rect x="145" y="182" width="600" height="12" rx="2" fill="#f59e0b" opacity="0.5" stroke="#f59e0b" stroke-width="0.5" stroke-dasharray="3,2"/>
  <text x="752" y="192" fill="#fbbf24" font-size="7">1.8 TB/s</text>

  <!-- NVSwitch topology -->
  <rect x="30" y="205" width="700" height="150" rx="6" fill="#080d14" stroke="#233043" stroke-width="1"/>
  <text x="380" y="225" fill="#e6edf3" font-size="10" font-weight="600" text-anchor="middle">NVSwitch All-to-All Topology (DGX H100)</text>

  <!-- 8 GPUs around NVSwitch -->
  <rect x="115" y="250" width="50" height="30" rx="3" fill="#0b1220" stroke="#34d399" stroke-width="0.8"/>
  <text x="140" y="269" fill="#34d399" font-size="7" text-anchor="middle">GPU 0</text>
  <rect x="195" y="250" width="50" height="30" rx="3" fill="#0b1220" stroke="#34d399" stroke-width="0.8"/>
  <text x="220" y="269" fill="#34d399" font-size="7" text-anchor="middle">GPU 1</text>
  <rect x="275" y="250" width="50" height="30" rx="3" fill="#0b1220" stroke="#34d399" stroke-width="0.8"/>
  <text x="300" y="269" fill="#34d399" font-size="7" text-anchor="middle">GPU 2</text>
  <rect x="355" y="250" width="50" height="30" rx="3" fill="#0b1220" stroke="#34d399" stroke-width="0.8"/>
  <text x="380" y="269" fill="#34d399" font-size="7" text-anchor="middle">GPU 3</text>
  <rect x="435" y="250" width="50" height="30" rx="3" fill="#0b1220" stroke="#34d399" stroke-width="0.8"/>
  <text x="460" y="269" fill="#34d399" font-size="7" text-anchor="middle">GPU 4</text>
  <rect x="515" y="250" width="50" height="30" rx="3" fill="#0b1220" stroke="#34d399" stroke-width="0.8"/>
  <text x="540" y="269" fill="#34d399" font-size="7" text-anchor="middle">GPU 5</text>
  <rect x="595" y="250" width="50" height="30" rx="3" fill="#0b1220" stroke="#34d399" stroke-width="0.8"/>
  <text x="620" y="269" fill="#34d399" font-size="7" text-anchor="middle">GPU 6</text>
  <rect x="675" y="250" width="50" height="30" rx="3" fill="#0b1220" stroke="#34d399" stroke-width="0.8"/>
  <text x="700" y="269" fill="#34d399" font-size="7" text-anchor="middle">GPU 7</text>

  <!-- NVSwitch chips (4 in H100) -->
  <rect x="200" y="300" width="360" height="25" rx="4" fill="#1c1633" stroke="#a78bfa" stroke-width="1"/>
  <text x="380" y="316" fill="#c4b5fd" font-size="8" text-anchor="middle">NVSwitch ×4 (all-to-all non-blocking crossbar)</text>

  <!-- Connection lines -->
  <line x1="140" y1="280" x2="260" y2="300" stroke="#a78bfa" stroke-width="0.5"/>
  <line x1="220" y1="280" x2="300" y2="300" stroke="#a78bfa" stroke-width="0.5"/>
  <line x1="300" y1="280" x2="340" y2="300" stroke="#a78bfa" stroke-width="0.5"/>
  <line x1="380" y1="280" x2="380" y2="300" stroke="#a78bfa" stroke-width="0.5"/>
  <line x1="460" y1="280" x2="420" y2="300" stroke="#a78bfa" stroke-width="0.5"/>
  <line x1="540" y1="280" x2="460" y2="300" stroke="#a78bfa" stroke-width="0.5"/>
  <line x1="620" y1="280" x2="500" y2="300" stroke="#a78bfa" stroke-width="0.5"/>
  <line x1="700" y1="280" x2="540" y2="300" stroke="#a78bfa" stroke-width="0.5"/>

  <text x="380" y="340" fill="#8b98a5" font-size="8" text-anchor="middle">any GPU can read/write any other GPU's HBM at 900 GB/s — unified 640 GB memory pool</text>
  <text x="380" y="354" fill="#6b7684" font-size="7.5" text-anchor="middle">enables 8-way tensor parallelism within a single node (critical for large model inference)</text>

  <!-- Bottom: why it matters -->
  <rect x="30" y="368" width="700" height="62" rx="6" fill="#080d14" stroke="#233043" stroke-width="1"/>
  <text x="380" y="388" fill="#e6edf3" font-size="10" font-weight="600" text-anchor="middle">Why NVLink Matters for AI</text>
  <text x="50" y="408" fill="#8b98a5" font-size="8.5">tensor parallelism sends activations between GPUs every layer → needs 900 GB/s (PCIe = 14× too slow)</text>
  <text x="50" y="422" fill="#6b7684" font-size="7.5">NVLink 5 (2025): 1.8 TB/s + NVLink Switch for 576 GPUs in one domain (GB200 NVL72 rack)</text>

  <text x="380" y="452" fill="#6b7684" font-size="11" text-anchor="middle">NVLink is NVIDIA's moat — it makes multi-GPU feel like one giant GPU, and competitors can't replicate it.</text>
</svg>

NVSwitch Fabric:

• Switch Architecture: NVSwitch is a dedicated switch chip providing full non-blocking connectivity among GPUs; each NVSwitch has 64 NVLink ports (NVSwitch 3.0 in H100 systems); multiple NVSwitches create a two-tier fabric for larger GPU counts
• DGX H100 Configuration: 8 H100 GPUs connected via 4 NVSwitches; every GPU has direct NVLink path to every other GPU; 900 GB/s bidirectional bandwidth between any GPU pair; total fabric bandwidth 7.2 TB/s
• Scalability: DGX SuperPOD connects 32 DGX H100 nodes (256 GPUs) using InfiniBand for inter-node and NVLink for intra-node; hybrid topology optimizes for locality (NVLink for nearby GPUs, IB for distant GPUs)
• Comparison to Direct Connection: without NVSwitch, 8 GPUs in ring/mesh topology have non-uniform bandwidth (adjacent GPUs: 900 GB/s, distant GPUs: 225-450 GB/s); NVSwitch provides uniform 900 GB/s between all pairs

Performance Characteristics:

• Bandwidth: NVLink 4.0 delivers 900 GB/s bidirectional per GPU; 14× higher than PCIe 5.0 x16 (64 GB/s); enables model parallelism where layer outputs (multi-GB activations) transfer between GPUs every forward/backward pass
• Latency: GPU-to-GPU load/store latency <1μs over NVLink vs 3-5μs over PCIe; low latency critical for fine-grained parallelism (tensor parallelism with frequent small transfers)
• CPU Overhead: NVLink transfers initiated by GPU without CPU involvement; cudaMemcpy() between peer GPUs uses NVLink automatically; zero CPU cycles consumed for GPU-to-GPU communication
• Coherency: NVLink supports cache-coherent memory access; GPU can cache remote GPU memory in its L2; reduces latency for repeated accesses to same remote data; coherency protocol ensures consistency across GPU caches

Programming Model:

• Peer Access: cudaDeviceEnablePeerAccess() enables direct addressing; GPU 0 can use device pointers from GPU 1 directly in kernels; cudaMemcpy() automatically uses NVLink for peer transfers
• Unified Memory: with NVLink, Unified Memory (cudaMallocManaged) provides single address space across GPUs; page migration and coherency handled by hardware/driver; simplifies multi-GPU programming but may have performance overhead from page faults
• NCCL Optimization: NCCL detects NVLink topology and uses optimized algorithms; ring all-reduce over NVLink achieves 95%+ of theoretical bandwidth; tree algorithms for NVSwitch topologies exploit full bisection bandwidth
• Explicit Topology Control: NCCL_TOPO_FILE environment variable specifies custom topology; enables manual optimization for non-standard configurations; useful for debugging performance issues or testing different communication patterns

Use Cases and Benefits:

• Model Parallelism: split large models (GPT-3, Megatron) across GPUs; layer outputs (activation tensors) transfer over NVLink every forward/backward pass; 900 GB/s enables model parallelism with <10% communication overhead
• Pipeline Parallelism: different layers on different GPUs; micro-batches flow through pipeline; NVLink bandwidth enables fine-grained pipelines (small micro-batches) with high throughput
• Data Parallelism: gradient all-reduce over NVLink; 8-GPU all-reduce completes in <1ms for billion-parameter models; enables large batch sizes (global batch = 8× per-GPU batch) without communication bottleneck
• Large Batch Training: NVLink enables efficient batch splitting across GPUs; each GPU processes subset of batch, exchanges activations/gradients; 900 GB/s supports batch sizes of 10,000+ images for vision models

Limitations and Considerations:

• Proprietary Technology: NVLink only connects NVIDIA GPUs; vendor lock-in limits flexibility; AMD Infinity Fabric and Intel Xe Link are competing technologies but less mature
• Distance Limitations: NVLink cables limited to ~2m; restricts GPU placement to single chassis or adjacent racks; inter-rack communication requires InfiniBand or Ethernet
• Cost: NVSwitch adds significant cost ($10K+ per switch); DGX systems with NVSwitch 2-3× more expensive than PCIe-only systems; cost justified only for workloads bottlenecked by GPU-to-GPU communication
• Topology Complexity: optimal NVLink topology depends on workload communication pattern; ring topology optimal for all-reduce, mesh for all-to-all, fully-connected (NVSwitch) for arbitrary patterns; misconfigured topology can leave bandwidth underutilized

NVLink is the interconnect that makes multi-GPU systems behave like single massive GPUs — by providing an order of magnitude more bandwidth than PCIe, NVLink enables model parallelism, large-batch training, and unified memory architectures that would be impractical with conventional interconnects, defining the architecture of modern AI supercomputers.