A DPU, or data processing unit, also sold as a SmartNIC, is the third class of processor in a modern datacenter, sitting alongside the CPU and the GPU. Where the CPU runs the application and the GPU runs the math, the DPU runs the infrastructure: the networking, storage, and security work that used to steal cycles from the host. Physically it is a network card with a full programmable system-on-chip bolted onto it, and its whole reason to exist is to take over the growing "datacenter tax" so that the expensive general-purpose cores and accelerators are freed to do the work a customer actually pays for.\n\nThe DPU exists to offload the datacenter tax that was eating host CPU cycles. As server networking climbed from ten to hundreds of gigabits per second, an ever-larger fraction of CPU time went not to the application but to moving packets, running the storage stack, encrypting traffic, and carrying the overhead of virtualization and the hypervisor. This infrastructure work is pure overhead from the application's point of view, and on a busy node it can consume a substantial share of the cores. The DPU takes that entire burden off the host processor.\n\nArchitecturally it is a NIC fused with a programmable SoC that runs its own operating system. On one board sit the high-speed network ports, a cluster of general-purpose CPU cores, usually Arm, a set of hardware accelerators for cryptography, compression, and packet and flow processing, a fast RDMA engine, and dedicated memory. Crucially the DPU boots and runs its own software stack independent of the host, so it is not just an accelerator the host calls but a small autonomous computer that sits between the server and the network.\n\nIt offloads three broad domains: networking, storage, and security. For networking it runs the virtual switch, RDMA and RoCE transport, and congestion control directly on the card. For storage it terminates NVMe-over-Fabrics so that remote disks across the network appear to the host as ordinary local drives. For security it does line-rate encryption and, because it is a separate trust domain from the host, enforces isolation that the host cannot tamper with, which is what makes secure bare-metal multi-tenancy and zero-trust models practical in the cloud.\n\nIn AI clusters the DPU becomes the intelligent edge of the fabric. Each GPU node's DPU manages the RDMA transfers that carry the all-reduce and all-to-all traffic of distributed training, enforces isolation between different tenants or jobs sharing the same cluster, and can accelerate parts of collective communication, complementing in-network reduction done on the switches. Every cycle it reclaims from infrastructure is a cycle of CPU or GPU compute sold to the customer, and the clean control-and-trust boundary it creates is exactly what a multi-tenant AI cloud needs.\n\n| Offload domain | What the DPU runs | What it frees the host from |\n|---|---|---|\n| Networking | Virtual switch, RDMA/RoCE, congestion control | Packet processing on host cores |\n| Storage | NVMe-over-Fabrics termination | Running the remote-storage stack |\n| Security | Line-rate encryption, isolation | Trusting the host for tenant isolation |\n| Management | Own OS, telemetry, provisioning | Host agents for infrastructure control |\n\n``svg\n\n``\n\nRead the DPU through an infrastructure-offload lens rather than a faster-network-card lens. Once you see that the CPU runs the app, the GPU runs the math, and the datacenter still has a third pile of work, moving packets, serving remote storage, encrypting traffic, isolating tenants, it becomes clear why that work wants its own processor sitting between the server and the network, reclaiming host cycles for paying compute and drawing a hard trust boundary that a multi-tenant AI cloud cannot do without.