Operating System Process is the fundamental unit of program execution that provides isolated memory, its own set of resources, and an independent execution context — the OS abstraction that enables multiprocessing in Python AI systems, provides crash isolation between services, and forms the basis of containerization in AI infrastructure.

What Is an OS Process?

• Definition: An instance of a running program consisting of: its own private virtual address space (memory), program counter, register state, open file handles, network connections, and at least one thread of execution — managed by the OS kernel.
• Isolation Guarantee: Process A cannot directly read or write Process B's memory — the kernel enforces virtual memory boundaries. A crash (segfault) in one process does not affect others.
• Process ID (PID): Every process has a unique integer identifier assigned by the OS. Used by ps, top, kill, and /proc/[pid]/ to monitor and manage processes.
• Creating Processes: On Unix/Linux, processes are created via fork() (copies current process) or exec() (replaces current process image with a new program). Python's subprocess and multiprocessing use these system calls.

Why Process Isolation Matters for AI Systems

• Model Serving Isolation: Running each model (embedding, reranker, LLM) as a separate process means a CUDA OOM in the LLM process cannot crash the embedding service.
• Worker Isolation in DataLoader: PyTorch DataLoader's worker processes are separate OS processes — a segfault in a preprocessing worker is caught by the DataLoader without crashing the training process.
• Container = Process: Docker containers are OS processes with namespace isolation (network, filesystem, PID) — understanding processes clarifies why containers are lightweight compared to VMs.
• Ray Actors: Ray's distributed computing abstraction maps directly to OS processes — each Ray actor is a Python process on a worker node, isolated from other actors.
• Gunicorn Workers: Production API servers (Gunicorn, uWSGI) spawn multiple worker processes — each handles requests independently, providing crash isolation and multi-core CPU utilization.

Process vs Thread Comparison

Process Life Cycle

Fork: Parent calls fork() → kernel creates identical child process (copy-on-write memory). Exec: Child optionally calls exec() to replace itself with a new program binary. Running: Process executes, makes system calls, uses CPU and memory. Waiting: Process blocks on I/O, sleep, or waiting for child (wait() system call). Zombie: Process has exited but parent has not yet called wait() to collect exit status. Terminated: Parent called wait() — OS reclaims all resources.

IPC (Inter-Process Communication) in AI

Since processes cannot share memory directly, they communicate via IPC:

Pipes/Queues: Byte streams between processes. from multiprocessing import Queue q = Queue() q.put(tensor.cpu().numpy()) # Serialize to queue data = q.get() # Deserialize in worker

Shared Memory: Zero-copy sharing of arrays (NumPy, tensors). from multiprocessing import shared_memory shm = shared_memory.SharedMemory(create=True, size=array.nbytes)

Zero-copy access from multiple processes

Sockets: TCP/UDP communication — used by Ray, gRPC, and REST APIs between services.

Memory-Mapped Files: Map a file into multiple processes' address spaces for zero-copy data access — used for large dataset sharing.

Process Management in AI Infrastructure

Supervisor / systemd: Manage long-running AI service processes — restart on crash, log output, manage environment.

Gunicorn: gunicorn app:app --workers 4 --worker-class uvicorn.workers.UvicornWorker Spawns 4 worker processes, each running the FastAPI/inference app — provides multi-core CPU utilization and crash isolation.

torch.multiprocessing: PyTorch's process pool with CUDA-aware shared memory — enables safe tensor sharing between training processes.

Kubernetes Pods: A pod contains one or more containers (processes) sharing a network namespace — the OS process model maps directly to Kubernetes deployment patterns.

OS processes are the fundamental isolation boundary of AI infrastructure — understanding how the kernel creates, isolates, and manages processes clarifies every aspect of container orchestration, DataLoader worker behavior, model serving architecture, and the multi-processing patterns that unlock true CPU parallelism in Python-based AI pipelines.