GPU Multi-Process

**GPU Multi-Process Service MPS** is **an NVIDIA GPU feature enabling multiple CPU processes to concurrently utilize GPU resources through time-slicing and context management — enabling higher GPU utilization by preventing GPU idleness during CPU process switching and improving throughput for workloads with many small GPU kernels**. GPU multi-process service addresses the limitation that traditional GPU execution isolates each CPU process with exclusive access to GPU, preventing concurrent execution of kernels from different processes and leaving GPU idle during context switch delays. The MPS system uses proxy connections where multiple processes communicate with single connection to GPU, with central MPS daemon managing GPU resource allocation and scheduling across connected processes. The concurrency level in MPS is limited by GPU architecture and resource constraints, with typical implementations supporting 16-32 concurrent process contexts depending on GPU generation. The performance characteristics of MPS depend on workload mixing and GPU resource availability, with processes having incompatible resource requirements potentially causing contention and reduced overall throughput. The isolation guarantees in MPS are reduced compared to exclusive process contexts, with multiple processes sharing execution resources and potentially exhibiting cache interference and other contention effects. The performance prediction with MPS is challenging due to dynamic scheduling and resource contention, requiring careful measurement and profiling to validate application performance with MPS enabled. The power efficiency improvements from MPS come from higher GPU utilization reducing idle time and associated power consumption, often resulting in significant energy savings despite slightly reduced per-application performance. **GPU multi-process service MPS enables concurrent GPU access by multiple CPU processes through resource sharing and scheduling, improving aggregate system throughput.**