openmp programming
**OpenMP (Open Multi-Processing)** is the **directive-based shared-memory parallel programming API that enables incremental parallelization of sequential C/C++/Fortran programs by inserting compiler pragmas — where a single `#pragma omp parallel for` can parallelize a loop across all available CPU cores with minimal code change, making it the most widely-used approach for shared-memory parallelism in scientific computing, simulation, and performance-critical applications**.
**Execution Model**
OpenMP follows the fork-join model:
- **Serial Region**: The master thread executes sequential code.
- **Parallel Region**: `#pragma omp parallel` forks a team of threads. Each thread gets a unique ID (omp_get_thread_num()).
- **Work Sharing**: Within a parallel region, work is distributed via constructs like `for` (loop iterations), `sections` (distinct code blocks), or `task` (dynamic tasks).
- **Barrier**: Implicit barrier at the end of each work-sharing construct. All threads synchronize before continuing.
**Key Directives**
```c
// Parallel loop — most common usage
#pragma omp parallel for schedule(dynamic, 64) reduction(+:sum)
for (int i = 0; i < N; i++) {
sum += compute(data[i]);
}
// Task parallelism — dynamic, irregular workloads
#pragma omp parallel
#pragma omp single
for (node* p = head; p; p = p->next) {
#pragma omp task firstprivate(p)
process(p);
}
#pragma omp taskwait
```
**Data Scoping**
- **shared**: Variable is shared among all threads (default for most variables). Programmer must ensure no data races.
- **private**: Each thread gets its own uninitialized copy.
- **firstprivate**: Private copy initialized from the master thread's value.
- **reduction**: Each thread accumulates into a private copy; results are combined at the barrier. Thread-safe accumulation without explicit atomics.
**Scheduling Strategies**
| Schedule | Distribution | Best For |
|----------|-------------|----------|
| static | Fixed chunks (N/P per thread) | Uniform work per iteration |
| dynamic | On-demand chunks from queue | Variable work per iteration |
| guided | Decreasing chunk sizes | Mixed uniform/variable |
| auto | Compiler/runtime choice | Let implementation decide |
**Advanced Features (OpenMP 5.0+)**
- **Target Offloading**: `#pragma omp target` offloads computation to GPUs and accelerators. Maps data between host and device memory.
- **SIMD**: `#pragma omp simd` directs the compiler to vectorize a loop using SIMD instructions.
- **Task Dependencies**: `#pragma omp task depend(in:x) depend(out:y)` creates a task DAG with data-flow dependencies.
- **Memory Model**: OpenMP defines a relaxed-consistency shared memory model. `#pragma omp flush` enforces memory consistency between threads when needed.
**OpenMP is the pragmatic on-ramp to parallel computing** — enabling performance-critical loops and algorithms to exploit multicore hardware through incremental, directive-based parallelization that preserves the readability and maintainability of the original sequential code.