nvlink

NVLink is NVIDIA's high-bandwidth interconnect that wires GPUs directly to one another so they can read and write each other's memory far faster than over the standard PCIe bus. Paired with NVSwitch, it forms a scale-up fabric: within a server, every GPU gets direct, high-speed, all-to-all access to every other, so a group of accelerators behaves much like a single large one with a shared, fast memory space rather than a loose cluster of separate cards.\n\n**It exists to break the PCIe bottleneck between GPUs.** Multi-GPU training and inference constantly exchange activations, gradients, and weight shards. Over PCIe that traffic crawls and serializes through the host, starving the accelerators. NVLink gives each GPU dedicated point-to-point links with roughly an order of magnitude more bandwidth than a PCIe slot, and NVSwitch connects all GPUs in a node without blocking, so any pair can talk at full rate simultaneously. The result is that collective operations finish fast enough to overlap with compute.\n\n**Scale-up (NVLink) versus scale-out (network) is the key distinction.** NVLink defines a scale-up domain — the tightly coupled GPUs inside one server or NVLink-connected rack that share the fast fabric. Beyond that boundary you scale out over PCIe and the datacenter network (InfiniBand or Ethernet), which is far slower per GPU. This bandwidth cliff at the node edge is precisely why communication-heavy techniques like tensor parallelism are confined inside the NVLink island, while looser methods (pipeline and data parallelism) span the slower links between nodes.\n\n| | NVLink / NVSwitch (scale-up) | PCIe / network (scale-out) |\n|---|---|---|\n| Domain | GPUs inside a node/rack | across nodes |\n| Bandwidth / GPU | very high (~10× PCIe) | much lower |\n| Topology | direct all-to-all | hierarchical, host-mediated |\n| Memory model | fast peer access, near-shared | message passing |\n| Hosts | tensor parallelism, all-reduce | pipeline, data parallelism |\n\n```svg\n\n```\n\n**It reshapes how large models are laid out across hardware.** Because the fabric determines where fast communication is available, model-parallel layout follows the topology: put the shards that must all-reduce every layer inside the NVLink domain, and place looser boundaries where only the slow network reaches. As NVLink generations raise per-GPU bandwidth and NVSwitch widens the non-blocking domain, more of the model can be treated as if it lived on one giant accelerator, pushing the scale-up boundary outward and reducing how often work has to cross the slow edge.\n\nRead NVLink through a quant lens rather than a 'faster cable' lens: it sets the bandwidth available inside the scale-up domain, and the whole parallelism plan is a bandwidth-matching exercise — put the collective-heavy shards where GB/s is high, and the point-to-point boundaries where it is low. The design question is how many GPUs fit inside the non-blocking NVLink island and at what per-GPU bandwidth, because that number decides how large a tensor-parallel group you can run before the network edge, not the math, becomes the ceiling.