checkpoint restart fault tolerance

**Checkpoint-Restart Fault Tolerance** — Mechanisms for periodically saving application state to stable storage so that computation can resume from a recent checkpoint rather than restarting from the beginning after a failure. **Coordinated Checkpointing** — All processes synchronize to create a globally consistent snapshot at the same logical time, ensuring no in-flight messages are lost. Blocking protocols pause computation during the checkpoint, providing simplicity at the cost of idle time. Non-blocking coordinated checkpointing uses Chandy-Lamport style markers to capture consistent state while processes continue executing. The coordination overhead scales with process count, making this approach challenging at extreme scale where checkpoint frequency must balance recovery cost against lost computation. **Uncoordinated and Communication-Induced Checkpointing** — Each process checkpoints independently without global synchronization, reducing checkpoint overhead but complicating recovery. The domino effect can force cascading rollbacks to the initial state if checkpoint dependencies form long chains. Communication-induced checkpointing forces additional checkpoints when message patterns would create problematic dependencies, bounding the rollback distance. Message logging complements uncoordinated checkpointing by recording received messages so that processes can replay communication during recovery without requiring sender rollback. **Incremental and Optimization Techniques** — Incremental checkpointing saves only memory pages modified since the last checkpoint, detected through OS page protection mechanisms or dirty-bit tracking. Hash-based deduplication identifies unchanged memory blocks across checkpoints, reducing storage and I/O requirements. Compression algorithms like LZ4 and Zstandard reduce checkpoint size with minimal CPU overhead. Multi-level checkpointing stores frequent lightweight checkpoints in local SSD or node-local burst buffers while periodically writing full checkpoints to the parallel file system, matching checkpoint frequency to failure probability at each level. **Implementation Frameworks and Tools** — DMTCP transparently checkpoints unmodified Linux applications by intercepting system calls and saving process state including open files and network connections. Berkeley Lab Checkpoint Restart (BLCR) operates at the kernel level for lower overhead. SCR (Scalable Checkpoint Restart) provides a library for applications to write checkpoints to node-local storage with asynchronous flushing to the parallel file system. VeloC offers a multi-level checkpointing framework optimized for leadership-class supercomputers with heterogeneous storage hierarchies. **Checkpoint-restart fault tolerance remains the primary resilience mechanism for long-running parallel applications, enabling productive use of large-scale systems where component failures are inevitable.**