nvlink
**NVLink** is **NVIDIA's high-bandwidth interconnect for GPU-to-GPU and GPU-to-CPU communication** — providing 600-900 GB/s bidirectional bandwidth compared to PCIe's 64 GB/s, enabling efficient multi-GPU scaling for large model training and inference.
**What Is NVLink?**
- **Definition**: Proprietary high-speed GPU interconnect.
- **Purpose**: Fast multi-GPU communication.
- **Bandwidth**: 10-14× faster than PCIe Gen5.
- **Use Cases**: Multi-GPU training, large model sharding.
**Why NVLink Matters**
- **Model Parallelism**: Large models span multiple GPUs.
- **Gradient Sync**: Training requires fast parameter updates.
- **Memory Pooling**: Access memory across GPUs.
- **Inference**: Large models need GPU sharding.
- **Scaling Efficiency**: Minimizes communication bottleneck.
**Bandwidth Comparison**
**Interconnect Speeds**:
```
Interconnect | Bandwidth (Bi-dir) | Generation
------------------|-------------------|------------
NVLink 4 (Hopper) | 900 GB/s | H100
NVLink 3 (Ampere) | 600 GB/s | A100
NVLink 2 (Volta) | 300 GB/s | V100
PCIe Gen5 | 64 GB/s (×16) | Current
PCIe Gen4 | 32 GB/s (×16) | Previous
InfiniBand NDR | 400 Gbps per port | Network
```
**Practical Impact**:
```
Operation | PCIe Gen5 | NVLink 4
-----------------------|--------------|----------
Copy 80GB (A100 mem) | 1.25 sec | 0.13 sec
Gradient sync (10GB) | 156 ms | 11 ms
AllReduce efficiency | 70-80% | 95%+
```
**NVLink Topologies**
**DGX H100 Topology**:
```svg
```
**Consumer NVLink** (RTX 4090):
```
3090: NVLink bridge, 2 GPUs
4090: No NVLink support
```
**NVSwitch**
**What It Enables**:
```
Without NVSwitch:
- Direct links only between neighbor GPUs
- Limited topology
With NVSwitch:
- All-to-all connectivity
- Full bisection bandwidth
- Any GPU reaches any GPU directly
```
**DGX Generations**:
```
System | GPUs | Topology | GPU-GPU BW
-------------|------|---------------------|------------
DGX A100 | 8 | NVSwitch (full) | 600 GB/s
DGX H100 | 8 | NVSwitch (full) | 900 GB/s
DGX GH200 | 256 | Grace Hopper + NVL | 900 GB/s
```
**Programming with NVLink**
**NCCL (NVIDIA Collective Communications Library)**:
```python
import torch
import torch.distributed as dist
# Initialize with NCCL backend (uses NVLink automatically)
dist.init_process_group(backend="nccl")
# AllReduce uses NVLink when available
tensor = torch.randn(1000, device="cuda")
dist.all_reduce(tensor) # Automatically uses NVLink
```
**Peer-to-Peer Memory Access**:
```cuda
// Enable P2P access between GPUs
cudaDeviceEnablePeerAccess(peer_device, 0);
// Direct memory access across NVLink
cudaMemcpyPeer(dst, dstDevice, src, srcDevice, size);
```
**Checking NVLink**:
```bash
# Check NVLink status
nvidia-smi nvlink -s
# Show topology
nvidia-smi topo -m
# NVLink utilization
nvidia-smi nvlink -g 0
```
**NVLink vs. PCIe Use Cases**
```
Use Case | Best Interconnect
----------------------|------------------
Single GPU inference | PCIe (sufficient)
Multi-GPU training | NVLink (essential)
Large model inference | NVLink (model sharding)
Consumer workstation | PCIe (NVLink limited)
Data center | NVLink + InfiniBand
```
NVLink is **essential infrastructure for multi-GPU AI** — without high-bandwidth interconnects, scaling to multiple GPUs becomes inefficient as communication overhead dominates, making NVLink critical for training large models and serving them across GPU clusters.