shared memory ipc
**POSIX Shared Memory and Memory-Mapped Files** are the **inter-process communication (IPC) mechanisms that allow multiple processes to access the same region of physical memory** — providing the fastest possible data sharing between processes on the same machine (zero-copy, no kernel involvement after setup), essential for high-performance computing, database engines, ML inference serving, and any application where microsecond-level IPC latency matters.
**Shared Memory vs. Other IPC**
| IPC Method | Latency | Throughput | Complexity |
|-----------|---------|-----------|------------|
| Shared memory (mmap/shm) | ~100 ns | Memory bandwidth | Medium |
| Unix domain socket | ~1-5 µs | ~5 GB/s | Low |
| TCP/IP (localhost) | ~10-50 µs | ~2-5 GB/s | Low |
| Pipe/FIFO | ~1-5 µs | ~3-5 GB/s | Low |
| Message queue (POSIX) | ~5-10 µs | ~1-3 GB/s | Medium |
**POSIX Shared Memory API**
```c
#include
#include
// Process A: Create shared memory
int fd = shm_open("/my_shm", O_CREAT | O_RDWR, 0666);
ftruncate(fd, 4096); // Set size
void *ptr = mmap(NULL, 4096, PROT_READ | PROT_WRITE,
MAP_SHARED, fd, 0);
// Write data
memcpy(ptr, data, sizeof(data));
// Process B: Attach to same region
int fd = shm_open("/my_shm", O_RDONLY, 0666);
void *ptr = mmap(NULL, 4096, PROT_READ, MAP_SHARED, fd, 0);
// Read data — zero copy, same physical pages
```
**Memory-Mapped Files**
```c
// Map a file into memory
int fd = open("large_dataset.bin", O_RDONLY);
void *data = mmap(NULL, file_size, PROT_READ,
MAP_PRIVATE, fd, 0);
// Access file as if it were memory
double value = ((double*)data)[1000000];
// OS handles page faults → loads from disk on demand
```
**Key Differences**
| Feature | shm_open (POSIX SHM) | mmap (file-backed) |
|---------|---------------------|--------------------|
| Backing | tmpfs (RAM only) | Filesystem (disk/SSD) |
| Persistence | Until shm_unlink or reboot | Persistent on disk |
| Size limit | Available RAM | Disk space |
| Use case | Fast IPC | Large dataset access, persistence |
| Survives reboot | No | Yes (file persists) |
**Synchronization**
- Shared memory has no built-in synchronization → multiple processes can corrupt data.
- Solutions:
- **POSIX semaphores**: sem_open/sem_wait/sem_post for mutual exclusion.
- **Atomic operations**: Lock-free algorithms using __atomic builtins.
- **Futex**: Fast userspace mutex (Linux) → no syscall in uncontended case.
- **Reader-writer locks**: pthread_rwlock in shared memory region.
**ML / Data Pipeline Usage**
- **PyTorch DataLoader**: Workers use shared memory to pass tensors to training process.
- **Ray / Plasma**: Object store backed by shared memory → zero-copy tensor sharing.
- **Inference serving**: Model weights in shared memory → multiple worker processes share one copy.
- **Redis**: Uses mmap for persistence (RDB/AOF), shared memory for module communication.
**Huge Pages for Performance**
- Default: 4 KB pages → many TLB misses for large shared regions.
- Huge pages (2 MB / 1 GB): Fewer TLB entries needed → 10-30% throughput improvement.
- Enable: mmap with MAP_HUGETLB or mount hugetlbfs.
- Critical for: Large ML models, HPC simulations, database buffer pools.
POSIX shared memory and memory-mapped files are **the foundation of zero-copy IPC on modern systems** — by allowing multiple processes to directly access the same physical memory pages without kernel-mediated data copies, they provide the highest possible throughput for local inter-process data sharing, making them indispensable for ML inference pipelines, database engines, and any high-performance system where data must flow between processes at memory bandwidth speeds.