memory consistency models

**Memory Consistency Models** — Memory consistency models define the rules governing the order in which memory operations from different processors become visible to each other, establishing the contract between hardware, compilers, and programmers for reasoning about shared-memory parallel programs. **Sequential Consistency** — The strictest intuitive model provides simple guarantees: - **Definition** — the result of any execution appears as if all operations from all processors were executed in some sequential order, preserving each processor's program order - **Intuitive Reasoning** — programmers can reason about concurrent programs as if operations were interleaved on a single processor, making correctness analysis straightforward - **Performance Cost** — enforcing sequential consistency prevents many hardware and compiler optimizations including store buffers, write combining, and instruction reordering - **Lamport's Formulation** — Leslie Lamport's original definition requires that operations appear to execute atomically and in an order consistent with each processor's program order **Relaxed Consistency Models** — Hardware relaxes ordering for performance: - **Total Store Order (TSO)** — used by x86 processors, TSO allows a processor to read its own writes early from the store buffer but maintains ordering between stores and between loads - **Partial Store Order (PSO)** — relaxes store-to-store ordering, allowing stores to different addresses to complete out of program order while maintaining store-to-load ordering - **Weak Ordering** — distinguishes between ordinary and synchronization operations, only guaranteeing ordering at synchronization points while allowing arbitrary reordering between them - **Release Consistency** — further refines weak ordering by distinguishing acquire operations (which prevent subsequent operations from moving before them) from release operations (which prevent preceding operations from moving after them) **Memory Fences and Barriers** — Explicit ordering instructions restore guarantees: - **Full Memory Fence** — prevents any reordering of loads and stores across the fence point, providing sequential consistency at the cost of pipeline stalls - **Store Fence** — ensures all preceding stores are visible before any subsequent stores, useful for publishing data structures that other threads will read - **Load Fence** — ensures all preceding loads complete before any subsequent loads execute, preventing speculative reads from returning stale values - **Acquire-Release Pairs** — acquire semantics on loads and release semantics on stores create happens-before relationships that are sufficient for most synchronization patterns **Language-Level Memory Models** — Programming languages define portable guarantees: - **C++11 Memory Model** — defines six memory ordering options from relaxed to sequentially consistent, giving programmers explicit control over ordering constraints on atomic operations - **Java Memory Model** — the happens-before relation defines visibility guarantees, with volatile variables and synchronized blocks establishing ordering between threads - **Data Race Freedom** — both C++ and Java guarantee sequential consistency for programs free of data races, simplifying reasoning for well-synchronized programs - **Compiler Ordering Constraints** — language memory models restrict compiler optimizations that could reorder or eliminate memory operations visible to other threads **Memory consistency models are fundamental to correct parallel programming, as misunderstanding the ordering guarantees provided by hardware and languages leads to subtle concurrency bugs that manifest only under specific timing conditions.**