GPU Multi-Tenancy is the sharing of a single physical GPU among multiple applications, users, or virtual machines, providing isolation, fairness, and efficient utilization of expensive GPU resources that would otherwise sit idle when any single workload cannot fully saturate the device.

GPUs are expensive ($10,000-$40,000+ for data center GPUs) yet many workloads — inference serving, interactive development, small training jobs — utilize only 10-30% of GPU capacity. Multi-tenancy enables cost-effective GPU sharing, which is critical for cloud providers and enterprise GPU clusters.

GPU Sharing Mechanisms:

Time-Slicing: The GPU scheduler context-switches between multiple applications, giving each a time quantum of full GPU access. Simple and universally available. Drawbacks: context switch overhead (~25 microseconds on modern GPUs), no memory isolation (potential interference), and bursty latency (applications wait their turn). Suitable for development and non-latency-sensitive workloads.

NVIDIA MPS (Multi-Process Service): A daemon that funnels multiple CUDA contexts through a single hardware context, enabling true spatial sharing where multiple processes' kernels execute concurrently on different SMs. Benefits: eliminates context switching overhead, enables fine-grained SM sharing, and supports CUDA streams from different processes. Limitations: limited error isolation (one process faulting affects others), no memory protection between processes, and fixed partitioning of SM resources.

MIG (Multi-Instance GPU): Available on NVIDIA A100, A30, H100. Hardware-level partitioning divides the GPU into up to 7 independent instances, each with dedicated SMs, memory, and L2 cache. Full hardware isolation — one instance's fault or performance behavior doesn't affect others. Each MIG instance appears as an independent GPU to software. Limitation: partition sizes are predefined (not arbitrary), and total partitions are limited.

Kubernetes GPU Scheduling: For GPU clusters, resource management integrates with orchestration: NVIDIA Device Plugin exposes GPUs as schedulable Kubernetes resources; GPU sharing extensions enable fractional GPU allocation (e.g., 0.5 GPU); topology-aware scheduling considers NVLink topology and NUMA affinity; priority-based preemption enables high-priority workloads to preempt low-priority GPU tenants.

Fairness and QoS: Multi-tenant GPU scheduling must ensure: fair share (each tenant receives proportional GPU time), latency SLO (inference workloads need bounded response time), memory isolation (one tenant cannot access or corrupt another's data), and admission control (reject workloads that would degrade existing tenants below their SLOs).

GPU multi-tenancy is transforming GPUs from dedicated single-user devices into shared infrastructure resources — enabling cloud-scale GPU economics where utilization approaching CPU-level sharing efficiency unlocks the full value of expensive accelerator hardware.