cache coherence

**Cache coherence** is the hardware protocol that ensures every CPU core in a multi-core processor sees a consistent view of shared memory — guaranteeing that when one core writes a value, all other cores that read the same address see the updated value, not a stale copy from their own cache. Without coherence, multi-threaded programs would silently produce wrong results because different cores would disagree on the contents of memory. Every multi-core chip from a phone SoC to a 100+ core server processor implements a coherence protocol in hardware, consuming 10–30% of on-chip interconnect bandwidth. **Why coherence is necessary.** Each core has private L1/L2 caches for speed. When core 0 writes address X to its L1, core 1's L1 might still hold the old value of X. Without coherence, core 1 reads stale data — a silent data corruption. The coherence protocol ensures either: (1) core 1's copy is invalidated before core 0 writes, or (2) core 0's write is propagated to core 1's cache. This must happen automatically in hardware (software-managed coherence is too slow and error-prone for general-purpose code). **The two dominant protocol families:** | Protocol | Mechanism | Scalability | Bandwidth | Used in | |---|---|---|---|---| | Snoopy (bus-based) | Every cache snoops a shared bus; broadcast invalidations | 2–8 cores | High (every write broadcasts) | Small multi-core (phone SoC, embedded) | | Directory-based | A directory tracks which caches hold each line; point-to-point messages | 8–1000+ cores | Lower (unicast, not broadcast) | Server CPUs, GPU, AI SoC | **MESI — the classic snoopy protocol.** Each cache line is in one of four states: - **M (Modified):** This cache has the only valid copy; it's been written. Must write back before another cache can read. - **E (Exclusive):** This cache has the only copy, but it matches memory. Can transition to M on write without bus traffic. - **S (Shared):** Multiple caches may hold this line (read-only). Must invalidate others before writing. - **I (Invalid):** Line is not in this cache (or has been invalidated). Variants: MOESI (adds Owned state — dirty shared), MESIF (adds Forward — one cache supplies data to requesters instead of memory). **Directory-based coherence — scaling to many cores.** A snoopy protocol broadcasts every invalidation to all caches — O(N) traffic per write. At 64+ cores this saturates the interconnect. Directory protocols maintain a bit-vector (or pointer list) per cache line in a distributed directory, recording which cores hold copies. Invalidation messages are sent only to the sharers — O(1) messages per write, scaling to hundreds of cores. AMD's MOESI/Probe-Filter and Intel's MESIF/snoop-filter are production directory variants. **Coherence overhead and performance impact:** - **Latency:** A write to a shared line requires invalidating remote copies (20–100 ns round-trip to remote cache/directory vs 1–4 ns for a local L1 hit). - **Bandwidth:** Coherence messages (requests, invalidations, data responses) consume 10–30% of NoC/interconnect bandwidth. - **False sharing:** When two cores write to different variables that happen to share a cache line (64 bytes), the line ping-pongs between cores — catastrophic performance. Padding data structures to cache-line boundaries avoids this. - **Scalability wall:** Beyond ~100–200 cores with coherence, the overhead grows faster than the compute benefit — which is why GPUs use a simpler memory model (no hardware coherence across SMs, software-managed shared memory). **Coherence in AI chips — a design choice.** AI accelerators face a coherence trade-off: - **CPUs** (Grace, Zen): full hardware coherence — required for running OS, general-purpose code, pointer-rich data structures. - **GPUs** (H100 SM clusters): L1 is per-SM non-coherent; L2 is shared and coherent within the GPU; cross-GPU coherence via NVLink (limited, software-managed). - **Custom AI ASICs** (TPU, Cerebras): often no coherence at all — the dataflow is statically scheduled, so software knows where data lives and moves it explicitly via DMA. This eliminates coherence overhead entirely but requires compiler support. ```svg ``` **Cache coherence and the CFS platform.** Coherence is the hardware mechanism that makes multi-core programming possible — it's what lets two threads on different cores share a data structure without explicit message passing. The CFS NoC keyword covers the on-chip network that carries coherence messages. The memory-controller keyword covers the DRAM interface that coherence protocol ultimately reads from on a miss. The computer-architecture keyword covers the full system context where coherence operates.