In-Network and Near-Network Computing is the distributed computing paradigm that offloads computation from host CPUs to network devices — smart NICs (SmartNICs), Data Processing Units (DPUs), and programmable switches — performing operations like collective communication, data filtering, encryption, and protocol processing at line rate within the network fabric itself, reducing host CPU load, cutting latency, and eliminating redundant data movement in data center and HPC environments.

Why Compute in the Network

In a conventional architecture, every network packet traverses: NIC → PCIe → CPU → memory → CPU → PCIe → NIC. The CPU spends 30-50% of its cycles on networking overhead (protocol processing, checksums, encryption) — cycles stolen from application computation. Offloading this work to the network device frees CPU cores and often reduces latency by eliminating the round-trip through the memory hierarchy.

SmartNIC / DPU Architecture

• NVIDIA BlueField DPU: An ARM CPU (8-16 cores) + RDMA-capable NIC + programmable packet processing pipeline on a single PCIe card. Runs a full Linux OS — can execute containers, security functions, and storage services independently of the host CPU.
• AMD/Pensando DPU: P4-programmable packet processing pipeline + ARM cores. Targets cloud infrastructure offload (OVS, IPsec, NVMe-oF).
• Intel IPU (Infrastructure Processing Unit): FPGA-based + Xeon cores for programmable network and storage offload.

Offload Capabilities

• RDMA (Remote Direct Memory Access): The NIC reads/writes remote machine's memory directly, bypassing both CPUs' operating systems. Latency: 1-2 μs (vs. 20-50 μs for TCP/IP). Bandwidth: 400 Gbps per port. InfiniBand (RDMA-native) and RoCE (RDMA over Converged Ethernet) are the protocols.
• In-Network Collective Operations: NVIDIA SHARP (Scalable Hierarchical Aggregation and Reduction Protocol) performs MPI allreduce operations within the InfiniBand switches. Gradient aggregation for distributed training completes in switch hardware at line rate, eliminating the standard ring/tree all-reduce communication pattern.
• GPUDirect RDMA: NIC transfers data directly to/from GPU memory without involving the CPU or system memory. Removes two unnecessary memory copies from the GPU communication critical path.
• Encryption/Decryption: IPsec, TLS, and MACsec at line rate (400 Gbps) without CPU involvement. Essential for encrypted data center traffic that would otherwise consume multiple CPU cores.

Programmable Switches

P4-programmable switches (Intel Tofino, AMD/Pensando) can execute simple programs on every packet traversing the switch at line rate (12.8 Tbps). Applications: in-network caching (NetCache), consensus protocols (NetPaxos), load balancing, and telemetry (INT — In-Band Network Telemetry).

Impact on Parallel Computing

In-network computing most impacts distributed training: SHARP reduces all-reduce latency by 2-7x compared to host-based NCCL. For 1000+ GPU training runs, this translates to 5-15% total training time reduction — saving days of GPU time worth hundreds of thousands of dollars.

In-Network Computing is the data center's shift from "move data to computation" to "move computation to data" — embedding processing capability throughout the network fabric to eliminate the bottleneck of routing every byte through host CPUs that have better things to do.