gpu virtualization
**GPU Virtualization and Multi-Instance GPU (MIG)** is the **technology enabling a single physical GPU to be partitioned into multiple isolated instances** — each with dedicated compute resources, memory, and memory bandwidth, allowing multiple users or workloads to share one GPU safely without interference, maximizing GPU utilization in cloud and enterprise environments.
**Why GPU Virtualization?**
- Many workloads don't need a full GPU: Inference serving, Jupyter notebooks, small training jobs.
- Without sharing: A user occupying an A100 at 10% utilization wastes 90% of a $10,000+ GPU.
- With MIG: Split one A100 into 7 isolated instances → 7 users, each with guaranteed resources.
**NVIDIA MIG (Multi-Instance GPU)**
- Available on: A100, A30, H100, H200 GPUs.
- Partitions GPU into up to **7 instances** (on A100/H100).
- Each instance gets:
- Dedicated SM (streaming multiprocessor) slices.
- Dedicated memory and memory bandwidth.
- Dedicated L2 cache partition.
- Fault isolation (one instance's error doesn't crash others).
**MIG Partition Profiles (A100 80GB)**
| Profile | GPU Memory | SMs | Use Case |
|---------|-----------|-----|----------|
| 1g.10gb | 10 GB | 14 SMs | Small inference |
| 2g.20gb | 20 GB | 28 SMs | Medium inference/training |
| 3g.40gb | 40 GB | 42 SMs | Large inference |
| 4g.40gb | 40 GB | 56 SMs | Medium training |
| 7g.80gb | 80 GB | 98 SMs | Full GPU (no partition) |
**Other GPU Sharing Approaches**
| Approach | Isolation | Overhead | Flexibility |
|----------|----------|---------|------------|
| MIG | Hardware-enforced | Near zero | Fixed profiles |
| vGPU (NVIDIA GRID) | Driver-level | 5-15% | Time-slicing |
| MPS (Multi-Process Service) | Software | Low | Concurrent kernels |
| Time-Slicing | Context switching | 10-30% | Any workload |
| Kubernetes GPU Sharing | Orchestration | Varies | Pod-level |
**vGPU (Virtual GPU)**
- NVIDIA GRID/vGPU: Hypervisor-based GPU virtualization.
- GPU time-sliced between VMs — each VM sees a virtual GPU.
- Used in: VDI (virtual desktops), cloud gaming, VMware/Citrix environments.
- Overhead: 5-15% per VM due to context switching.
**MPS (Multi-Process Service)**
- Allows multiple CUDA processes to share a single GPU simultaneously.
- Processes run concurrently (not time-sliced) — better utilization than context switching.
- No memory isolation — one process can potentially access another's memory.
- Used when: trusted workloads need to share GPU without MIG overhead.
**Cloud GPU Sharing**
- AWS: `p4d.24xlarge` with 8 A100s, or MIG-backed instances.
- GCP: Multi-instance GPU support for A100/H100.
- Azure: MIG available on ND-series VMs.
GPU virtualization is **essential for economic GPU utilization in data centers** — without partitioning and sharing, the high cost of modern GPU accelerators would be wasted on workloads that use only a fraction of available compute and memory resources.