rdma infiniband
**RDMA (Remote Direct Memory Access) and InfiniBand** are the **high-performance networking technologies that enable direct memory-to-memory data transfer between machines without involving the CPU or operating system** — achieving latencies under 1 microsecond and throughputs over 400 Gbps, making them essential for HPC clusters, distributed training, and low-latency storage systems.
**How RDMA Works**
- **Traditional networking**: App → OS kernel → TCP/IP stack → NIC → wire → NIC → kernel → App.
- Each step: System calls, context switches, memory copies — adds latency.
- **RDMA**: App → NIC → wire → NIC → remote memory (bypasses both CPUs and kernels).
- **Zero-copy**: Data goes directly from wire to application buffer — no intermediate copies.
- **Kernel bypass**: NIC handles protocol processing in hardware — no OS involvement.
- **CPU offload**: CPU freed for computation while NIC handles transfers.
**RDMA Operations**
| Operation | Description | CPU Involvement |
|-----------|-------------|----------------|
| RDMA Write | Write to remote memory | None on remote side |
| RDMA Read | Read from remote memory | None on remote side |
| Send/Receive | Two-sided messaging | Both sides post buffers |
| Atomic (CAS, FetchAdd) | Atomic operation on remote memory | None on remote side |
**RDMA Transports**
| Transport | Fabric | Bandwidth | Latency | Deployment |
|-----------|--------|-----------|---------|------------|
| InfiniBand (IB) | Dedicated IB fabric | HDR: 200 Gbps, NDR: 400 Gbps | < 0.6 μs | HPC, AI clusters |
| RoCE v2 | Standard Ethernet | 25-400 Gbps | 1-3 μs | Data centers |
| iWARP | Standard Ethernet (TCP) | 10-100 Gbps | 5-10 μs | Enterprise storage |
**InfiniBand Generations**
| Generation | Per-Lane Rate | 4x Port | Year |
|-----------|-------------|---------|------|
| QDR | 10 Gbps | 40 Gbps | 2008 |
| FDR | 14 Gbps | 56 Gbps | 2012 |
| EDR | 25 Gbps | 100 Gbps | 2015 |
| HDR | 50 Gbps | 200 Gbps | 2019 |
| NDR | 100 Gbps | 400 Gbps | 2022 |
| XDR | 200 Gbps | 800 Gbps | 2024 |
**RDMA in Distributed ML Training**
- **NCCL over InfiniBand**: Default for multi-node GPU training.
- GPUDirect RDMA: NIC reads directly from GPU memory — no CPU staging buffer.
- 8× H100 DGX pods connected via 8× NDR400 (3.2 Tbps per node).
- **Gradient AllReduce**: Ring/tree AllReduce over RDMA achieves near-wire-speed.
- Without RDMA: Multi-node training bandwidth drops 5-10x → scaling becomes impractical beyond 2-4 nodes.
**Programming RDMA**
- **libibverbs**: Low-level C API for RDMA operations (complex — ~200 LOC for simple send).
- **UCX (Unified Communication X)**: Higher-level library abstracting RDMA transports.
- **NCCL / Gloo**: ML-specific collective communication over RDMA.
RDMA and InfiniBand are **the networking foundation of modern AI supercomputers** — the ability to move data between machines at hardware speed without CPU involvement is what makes it possible to train trillion-parameter models across thousands of GPUs with near-linear scaling efficiency.