distributed shared memory consistency
**Distributed Shared Memory (DSM) and Consistency Models** define **how memory operations across multiple processors are ordered and made visible to other processors**, establishing the contract between hardware/system software and the programmer about when a write by one processor will be seen by a read from another — a fundamental concern that affects both correctness and performance of parallel programs.
In shared-memory multiprocessors (including multi-core CPUs and NUMA systems), the memory consistency model determines what reorderings of memory operations are permitted. Stronger models are easier to program but limit hardware optimization; weaker models enable higher performance but require explicit synchronization.
**Memory Consistency Models**:
| Model | Ordering Guarantee | Performance | Programmability |
|-------|-------------------|------------|----------------|
| **Sequential Consistency** | All ops in total order respecting program order | Lowest | Easiest |
| **TSO (Total Store Order)** | Stores ordered, reads may pass stores | Good | Moderate |
| **Relaxed (ARM, POWER)** | Almost no ordering without fences | Best | Hardest |
| **Release Consistency** | Ordering only at acquire/release points | Good | Moderate |
**Sequential Consistency (SC)**: Lamport's model — the result of any execution is as if all operations were executed in some sequential order, and the operations of each processor appear in program order. SC is the most intuitive model but prevents hardware optimizations: store buffers, write combining, and out-of-order memory access are all restricted.
**Total Store Order (TSO)**: Used by x86/x64. All stores are ordered and seen by all processors in the same order. However, a processor may read its own store before it becomes visible to others (store buffer forwarding). This means: reads can be reordered before earlier stores to different addresses. Most SC programs work correctly under TSO, but subtle bugs can arise with flag-based synchronization (requiring MFENCE or locked instructions).
**Relaxed Models (ARM, RISC-V)**: Allow virtually all reorderings: loads reordered with loads, stores with stores, loads with stores. The programmer must insert explicit **memory barriers** (DMB/DSB on ARM, fence on RISC-V) to enforce ordering. C/C++ atomics abstract over hardware models: `memory_order_acquire`, `memory_order_release`, `memory_order_seq_cst` generate appropriate barriers for each architecture.
**Cache Coherence Protocols**: Hardware maintains the illusion that each memory location has a single, consistent value across all caches. **MESI protocol** (Modified, Exclusive, Shared, Invalid) tracks cache line state: before writing, a core must obtain exclusive ownership (invalidating all other copies). **MOESI** adds Owned state (dirty shared copy, avoids writeback). **Directory-based** protocols (used in NUMA/many-core) use a central directory to track which caches hold each line, avoiding broadcast snoops that don't scale beyond ~64 cores.
**DSM Systems**: Distributed Shared Memory extends the shared-memory abstraction across physically distributed machines: software DSM (Treadmarks, JIAJIA) uses page-fault handlers to implement remote memory access transparently; hardware DSM (SGI Origin, nowadays CXL) provides hardware-supported remote memory access. Modern CXL (Compute Express Link) memory expanders enable hardware-coherent DSM across PCIe-attached memory pools.
**Memory consistency models are the invisible contract that governs concurrent programming correctness — an algorithm that works perfectly on x86 (TSO) may fail silently on ARM (relaxed) due to reordering, making consistency model awareness essential for writing portable parallel software.**