network on chip noc

**Network-on-Chip (NoC) Architecture** is a **scalable on-chip communication framework that replaces traditional bus-based interconnects with packet-switched networks, enabling efficient data movement in many-core and AI accelerator chips.** **NoC Topology and Routing** - **Mesh Topology**: Regular 2D grid arrangement of routers (most common). Scales well to moderate core counts (~100s cores) with predictable performance. - **Torus Topology**: Mesh with wrap-around connections on edges. Reduces diameter and improves bisection bandwidth compared to mesh. - **Ring Topology**: Linear ordering of nodes. Lower area overhead but higher latency for distant cores. - **Routing Algorithms**: XY routing (dimension-ordered), adaptive routing selects alternate paths based on congestion. Deadlock-free routing using virtual channels. **NoC Router Microarchitecture** - **Input/Output Port Design**: Each router port includes input buffers (FIFO), crossbar switch, and arbitration logic. - **Virtual Channels**: Multiple independent channels per physical link prevent HOL (head-of-line) blocking and enable deadlock avoidance. Typically 4-8 VCs per port. - **Crossbar Switch**: Handles simultaneous transfers between input and output ports. Area and power scale as O(n²) where n is radix. - **Arbiter Implementations**: Round-robin, priority-based, or weighted arbitration for port conflicts. Critical for throughput and fairness. **Flow Control and QoS** - **Wormhole Switching**: Packet travels in flits. Low latency, low buffer overhead but entire packet remains in-flight during routing. - **Virtual Cut-Through**: Buffers entire packet at intermediate nodes. Higher latency but enables better path optimization. - **QoS Mechanisms**: Traffic class assignment, priority levels, bandwidth reservation for real-time tasks (critical for SoC interconnects). **Real-World Usage and Performance** - **Many-Core CPUs**: 64+ core designs require NoC for intra-cluster and inter-cluster communication. - **AI Accelerators**: Tensor cores demand low-latency, high-bandwidth communication. TPU, Cerebras, and Graphcore use custom NoC designs. - **Typical Performance**: 5-10 cycle latency per hop in modern implementations. Throughput limited by virtual channel bandwidth and arbitration efficiency.