process
**Operating System Process** is the **fundamental unit of program execution that provides isolated memory, its own set of resources, and an independent execution context** — the OS abstraction that enables multiprocessing in Python AI systems, provides crash isolation between services, and forms the basis of containerization in AI infrastructure.
**What Is an OS Process?**
- **Definition**: An instance of a running program consisting of: its own private virtual address space (memory), program counter, register state, open file handles, network connections, and at least one thread of execution — managed by the OS kernel.
- **Isolation Guarantee**: Process A cannot directly read or write Process B's memory — the kernel enforces virtual memory boundaries. A crash (segfault) in one process does not affect others.
- **Process ID (PID)**: Every process has a unique integer identifier assigned by the OS. Used by ps, top, kill, and /proc/[pid]/ to monitor and manage processes.
- **Creating Processes**: On Unix/Linux, processes are created via fork() (copies current process) or exec() (replaces current process image with a new program). Python's subprocess and multiprocessing use these system calls.
**Why Process Isolation Matters for AI Systems**
- **Model Serving Isolation**: Running each model (embedding, reranker, LLM) as a separate process means a CUDA OOM in the LLM process cannot crash the embedding service.
- **Worker Isolation in DataLoader**: PyTorch DataLoader's worker processes are separate OS processes — a segfault in a preprocessing worker is caught by the DataLoader without crashing the training process.
- **Container = Process**: Docker containers are OS processes with namespace isolation (network, filesystem, PID) — understanding processes clarifies why containers are lightweight compared to VMs.
- **Ray Actors**: Ray's distributed computing abstraction maps directly to OS processes — each Ray actor is a Python process on a worker node, isolated from other actors.
- **Gunicorn Workers**: Production API servers (Gunicorn, uWSGI) spawn multiple worker processes — each handles requests independently, providing crash isolation and multi-core CPU utilization.
**Process vs Thread Comparison**
| Aspect | Process | Thread |
|--------|---------|--------|
| Memory space | Private, isolated | Shared with parent process |
| Creation cost | High (fork ~1ms) | Low (~microseconds) |
| Memory overhead | High (full copy of address space) | Low (shared pages) |
| Crash isolation | Yes — crash doesn't affect others | No — crash kills entire process |
| Data sharing | IPC required (pipes, queues, shared memory) | Direct (but needs locks) |
| GIL | Each process has its own GIL | Shared GIL — no true parallelism for Python |
| Use case | CPU-bound parallelism | I/O-bound concurrency |
**Process Life Cycle**
Fork: Parent calls fork() → kernel creates identical child process (copy-on-write memory).
Exec: Child optionally calls exec() to replace itself with a new program binary.
Running: Process executes, makes system calls, uses CPU and memory.
Waiting: Process blocks on I/O, sleep, or waiting for child (wait() system call).
Zombie: Process has exited but parent has not yet called wait() to collect exit status.
Terminated: Parent called wait() — OS reclaims all resources.
**IPC (Inter-Process Communication) in AI**
Since processes cannot share memory directly, they communicate via IPC:
**Pipes/Queues**: Byte streams between processes.
from multiprocessing import Queue
q = Queue()
q.put(tensor.cpu().numpy()) # Serialize to queue
data = q.get() # Deserialize in worker
**Shared Memory**: Zero-copy sharing of arrays (NumPy, tensors).
from multiprocessing import shared_memory
shm = shared_memory.SharedMemory(create=True, size=array.nbytes)
# Zero-copy access from multiple processes
**Sockets**: TCP/UDP communication — used by Ray, gRPC, and REST APIs between services.
**Memory-Mapped Files**: Map a file into multiple processes' address spaces for zero-copy data access — used for large dataset sharing.
**Process Management in AI Infrastructure**
**Supervisor / systemd**: Manage long-running AI service processes — restart on crash, log output, manage environment.
**Gunicorn**:
gunicorn app:app --workers 4 --worker-class uvicorn.workers.UvicornWorker
Spawns 4 worker processes, each running the FastAPI/inference app — provides multi-core CPU utilization and crash isolation.
**torch.multiprocessing**: PyTorch's process pool with CUDA-aware shared memory — enables safe tensor sharing between training processes.
**Kubernetes Pods**: A pod contains one or more containers (processes) sharing a network namespace — the OS process model maps directly to Kubernetes deployment patterns.
OS processes are **the fundamental isolation boundary of AI infrastructure** — understanding how the kernel creates, isolates, and manages processes clarifies every aspect of container orchestration, DataLoader worker behavior, model serving architecture, and the multi-processing patterns that unlock true CPU parallelism in Python-based AI pipelines.