Interconnect Topology is the physical and logical arrangement of network links connecting compute nodes in parallel systems — determining the bandwidth, latency, scalability, and cost characteristics of the communication fabric that enables thousands to millions of processors to work together, with topology choice directly impacting application performance by 2-5x for communication-heavy workloads.

Common Topologies

Fat Tree (Clos Network)

• Structure: Multi-level tree with increasing bandwidth at each level → "fat" at top.
• Full bisection bandwidth: Any half of nodes can communicate with other half at full speed.
• Implementation: Standard Ethernet/InfiniBand switches in leaf-spine-core layers.
• Pros: Non-blocking, any-to-any communication at full bandwidth.
• Cons: Expensive — top-level switches carry all traffic. Cable count: O(N log N).

3D Torus

• Structure: Each node connected to 6 neighbors (±x, ±y, ±z). Wrap-around links at edges.
• IBM Blue Gene/Q: 5D torus with 10 links per node.
• Fujitsu Fugaku (#1 in 2020): 6D mesh/torus (Tofu-D interconnect).
• Pros: Simple, low cost (O(N) links), good for nearest-neighbor communication (stencil patterns).
• Cons: Low bisection bandwidth — all-to-all communication suffers.

Dragonfly

• Structure: Three levels — intra-group (local), inter-group (global), inter-cabinet.
• Groups: Fully connected internally. Groups connected by global links.
• Adaptive routing: Traffic dynamically routed to avoid congestion.
• Cray Slingshot / HPE Cray EX: Modern HPC systems use Dragonfly variants.
• Pros: Good balance of cost, bandwidth, and latency for diverse traffic patterns.

NVLink/NVSwitch Topology (GPU clusters)

• DGX A100 (8 GPUs): Full NVSwitch mesh — any GPU to any GPU at 600 GB/s.
• DGX H100 (8 GPUs): NVSwitch 4th gen — 900 GB/s per GPU.
• NVLink Network (multi-node): NVLink extended across nodes → GPU-to-GPU without CPU.

Routing Algorithms

• Deterministic: Same path for same source-destination → simple, may cause congestion.
• Adaptive: Route based on current congestion → better utilization, harder to implement.
• Minimal: Shortest path only. Non-minimal: May take longer paths to avoid congestion.

Interconnect topology is a defining architectural choice for any large-scale parallel system — the topology determines the communication performance envelope within which all parallel algorithms must operate, making it one of the first and most consequential decisions in supercomputer and data center design.