A network on chip (NoC) is the packet-switched communication fabric that moves data among processors, accelerators, caches, memory controllers, and I/O blocks inside a system on chip. It replaces the shared buses that worked for a handful of masters but become a timing, bandwidth, and arbitration bottleneck as an SoC grows. A NoC divides long global communication into short registered links, routes transactions through distributed switches, and lets many unrelated transfers proceed at once. The result is not merely wiring infrastructure: topology, routing, buffering, and quality-of-service policy directly determine application throughput, latency, power, and whether independent IP blocks can safely share the chip.

The central scaling idea is spatial reuse. On a bus, every participant competes for the same electrical and protocol resource. In a mesh, a packet traveling east can use different links at the same time that another packet travels north elsewhere. A wide AI accelerator may therefore sustain many terabytes per second of aggregate on-chip traffic even though no single link carries that total. Designers quote both link bandwidth and bisection bandwidth, the sum of capacity crossing a cut through the network. Bisection bandwidth is often the more revealing limit for all-to-all exchanges, cache-coherence traffic, or data movement between compute tiles and distributed SRAM.

A packet is broken into flow-control digits, usually called flits. The head flit carries routing and transaction metadata; body flits carry addresses or data; the tail releases resources. With wormhole switching, a packet occupies a sequence of small buffers and links rather than waiting for the whole packet at every router. That reduces buffer area and often reduces unloaded latency, but a blocked head flit can hold resources behind it. Virtual channels place several logical queues over one physical link so an obstructed traffic class does not necessarily block every other class.

A practical router contains input buffers, route computation, virtual-channel allocation, switch allocation, a crossbar, and registered output links. Route computation chooses an allowed next hop. Allocation arbitrates when several inputs request the same output. The crossbar connects winners for that cycle, and pipeline registers limit the wire length seen by static timing analysis. A three- or four-stage router may run faster than a single-cycle router but adds a cycle at every hop. High-radix routers reduce hop count while increasing crossbar, arbitration, and port wiring cost.

<svg viewBox="0 0 960 380" xmlns="http://www.w3.org/2000/svg" role="img" aria-labelledby="nocTitle nocDesc">
  <title id="nocTitle">Mesh network on chip connecting compute, cache, and memory tiles</title>
  <desc id="nocDesc">A four by three mesh of routers with example packets moving between CPU, AI, SRAM, and HBM controller endpoints.</desc>
  <rect width="960" height="380" rx="18" fill="#0b1017"/>
  <text x="480" y="32" text-anchor="middle" fill="#f1f5f9" font-size="20" font-weight="700">Tiled SoC with a packet-switched mesh</text>
  <g stroke="#334155" stroke-width="8" stroke-linecap="round">
    <path d="M170 105H790 M170 190H790 M170 275H790"/>
    <path d="M170 105V275 M375 105V275 M585 105V275 M790 105V275"/>
  </g>
  <g fill="#132235" stroke="#60a5fa" stroke-width="2">
    <rect x="125" y="68" width="90" height="74" rx="10"/><rect x="330" y="68" width="90" height="74" rx="10"/>
    <rect x="540" y="68" width="90" height="74" rx="10"/><rect x="745" y="68" width="90" height="74" rx="10"/>
    <rect x="125" y="153" width="90" height="74" rx="10"/><rect x="330" y="153" width="90" height="74" rx="10"/>
    <rect x="540" y="153" width="90" height="74" rx="10"/><rect x="745" y="153" width="90" height="74" rx="10"/>
    <rect x="125" y="238" width="90" height="74" rx="10"/><rect x="330" y="238" width="90" height="74" rx="10"/>
    <rect x="540" y="238" width="90" height="74" rx="10"/><rect x="745" y="238" width="90" height="74" rx="10"/>
  </g>
  <g fill="#dbeafe" font-size="12" text-anchor="middle" font-weight="600">
    <text x="170" y="101">CPU</text><text x="375" y="101">L3 slice</text><text x="585" y="101">AI tile</text><text x="790" y="101">SRAM</text>
    <text x="170" y="186">DSP</text><text x="375" y="186">Router</text><text x="585" y="186">Router</text><text x="790" y="186">I/O</text>
    <text x="170" y="271">GPU</text><text x="375" y="271">SRAM</text><text x="585" y="271">AI tile</text><text x="790" y="271">HBM ctrl</text>
  </g>
  <g fill="#34d399" stroke="#052e25" stroke-width="2"><circle cx="375" cy="105" r="9"/><circle cx="585" cy="105" r="9"/><circle cx="585" cy="190" r="9"/><circle cx="585" cy="275" r="9"/><circle cx="790" cy="275" r="9"/></g>
  <text x="480" y="345" text-anchor="middle" fill="#94a3b8" font-size="14">Registered links carry flits; routers select paths and isolate traffic with virtual channels.</text>
</svg>

Flow control prevents a sender from overwriting a full receiver. Credit-based flow control gives the upstream router a count of free downstream buffer entries. Sending a flit consumes a credit, and returning a credit reports that space has been released. Ready-valid handshakes are simpler over short links, while credits tolerate additional pipeline delay without stopping every round trip. Designers size buffers against credit latency and burst behavior: too little buffering wastes link cycles, while too much consumes leakage power and precious SRAM-like area.

Routing must balance efficiency with freedom from deadlock. Deterministic dimension-order routing, such as moving in X before Y, is easy to verify and creates predictable paths. Adaptive routing can steer around congestion or failed links, but it requires congestion information and careful rules. Deadlock occurs when packets form a cycle of resource dependencies and none can advance. Architects break those cycles by restricting turns, providing an escape virtual channel with deadlock-free routing, or separating protocol request and response traffic onto independent virtual networks.

Transaction ordering sits above packet delivery. AXI, CHI, TileLink, or a proprietary coherent protocol may require some operations to remain ordered while allowing unrelated identifiers to complete out of order. The network can preserve ordering by keeping flows on one path, tagging and reordering responses at endpoints, or constraining adaptive routing. Coherent systems also carry snoops, probes, acknowledgments, and data responses. Separating those message classes prevents a response needed to release a request from being trapped behind more requests.

Quality of service converts business priorities into arbitration rules. Display refresh, audio, safety traffic, and real-time control need bounded service; CPUs prefer low latency; bulk DMA and AI tensors prefer sustained bandwidth. Weighted round-robin, age-based priority, reserved virtual channels, and rate limiters are common tools. Strict priority alone is dangerous because low-priority traffic can starve. Verification must show minimum bandwidth and maximum latency under adversarial combinations, not merely good averages on representative software.

Performance analysis begins with offered load and locality. If average packet size is \(S\) bytes, injection rate is \(r\) packets per cycle, and clock frequency is \(f\), one endpoint offers \(B=rSf\) bytes per second. The links on its routes must collectively absorb that traffic. Latency remains close to router pipeline plus serialization delay at low utilization, then rises sharply near saturation as queues build. Synthetic uniform, hotspot, transpose, and burst traffic reveal structural limits; application traces reveal whether mapping and tiling create avoidable hot links.

AI chips make NoC design inseparable from dataflow. A matrix engine may consume hundreds of operands per cycle, but most useful reuse occurs in local registers or SRAM. The NoC should carry each tensor tile only when it changes ownership, then multicast weights or activations where possible. Hardware multicast saves repeated link traffic, while reduction support can combine partial sums near their sources. Mapping software needs a faithful cost model because placing communicating operators on distant tiles can turn arithmetic-rich silicon into a network-bound machine.

Physical implementation often changes the architectural optimum. Long links need repeaters or pipeline stages; dense router crossings compete with clock trees and power straps; wide links consume upper-metal tracks. A theoretically elegant crossbar can become unroutable, while a mesh aligns naturally with replicated tiles. Designers may use express links for frequent distant pairs, bridge separate voltage or clock domains, and place network interfaces at IP boundaries. Mesochronous or asynchronous crossings require synchronizers, elastic buffers, and reset sequences that do not drop credits.

Power is spent in buffers, arbitration logic, clocking, and wire transitions. Clock gating idle ports, narrowing links, reducing unnecessary hops, and encoding links can help, but each choice affects wake latency or throughput. Dynamic voltage and frequency scaling may create islands whose link capacity changes at runtime. Thermal throttling can similarly turn a once-balanced route into a hotspot, so robust systems coordinate NoC policy with power management rather than treating the fabric as fixed plumbing.

Reliability provisions range from parity to graceful degradation. Link CRC or parity detects corrupted flits; replay recovers transient errors; ECC protects deeper buffers. Timeout and poison mechanisms prevent silent hangs. Large chips may include spare links, disable a faulty router port, or update routing tables around manufacturing defects. These mechanisms need end-to-end validation because a retry can violate ordering and a reroute can introduce a dependency cycle that was absent from the nominal topology.

NoC verification combines formal proofs, constrained-random simulation, emulation, and performance modeling. Formal methods are well suited to local credit invariants, no-drop/no-duplicate properties, arbitration fairness, and selected deadlock arguments. Simulation stresses protocol ordering and reset. Emulation runs long software workloads. Performance models explore topology and buffer parameters before RTL stabilizes. Useful observability includes per-port counters, queue high-water marks, latency histograms, trace triggers, and packet error registers; without them, a workload slowdown can be nearly impossible to distinguish from memory or compute backpressure.

A good network on chip is judged by delivered system work, not an impressive aggregate bandwidth number. It must meet timing after placement, sustain critical traffic under contention, preserve the memory model, recover from errors, remain debuggable, and do so within area and power budgets. The best topology is therefore workload- and floorplan-specific. Architects succeed when software placement, protocol behavior, router microarchitecture, and physical wires are designed as one system.