GPU Virtual Memory Management is the system of hardware and software mechanisms that provide GPUs with virtual address spaces, demand paging, memory oversubscription, and unified addressing — evolving GPU memory from simple physical allocation to sophisticated virtual memory systems comparable to CPU memory management.

Historically, GPU memory was managed as a simple physical allocator: applications allocated fixed-size buffers in GPU VRAM, and any overflow required manual data staging through host memory. Modern GPUs provide full virtual memory support that fundamentally changes programming models.

Unified Virtual Addressing (UVA): CUDA's UVA (since CUDA 4.0) maps CPU and GPU memory into a single virtual address space. Any pointer can be dereferenced by either CPU or GPU — the runtime determines the physical location and handles data migration. This eliminates the need for separate host/device pointer management.

CUDA Unified Memory: Building on UVA, unified memory (managed memory) provides automatic page migration between CPU and GPU on demand. When the GPU accesses a page resident in CPU memory, a page fault triggers migration to GPU VRAM (and vice versa). The page fault mechanism (available since Pascal/sm_60) enables: memory oversubscription — GPU kernels can access more memory than physical VRAM by paging to system memory; simplified programming — no explicit cudaMemcpy calls; and prefetch hints — cudaMemPrefetchAsync allows applications to guide the migration system.

GPU Page Table Architecture: Modern GPUs (NVIDIA Ampere and later) implement multi-level page tables similar to CPU MMUs. GPU page sizes are typically larger (64KB-2MB versus CPU's 4KB-2MB) to amortize TLB miss overhead and match GPU's coalesced access patterns. GPU TLBs are organized per-SM with L1 TLB and shared L2 TLB. TLB misses are expensive on GPUs because they stall thousands of threads simultaneously.

Memory Oversubscription: When GPU VRAM is exhausted, pages are evicted to system memory. The GPU runtime implements a page replacement policy (LRU-based or access-frequency-based). Performance degrades as oversubscription increases because: PCIe/NVLink bandwidth (32-900 GB/s) is far below GPU memory bandwidth (~3 TB/s), and page faults stall warps until migration completes. However, oversubscription enables running workloads that previously required model sharding or data streaming.

Access Counters and Prefetching: Hardware access counters track page access frequency and locality. The driver uses this telemetry for intelligent page placement: frequently-accessed pages migrate to VRAM, cold pages demote to system memory. Prefetching algorithms predict future access patterns (based on sequential detection or application hints) and migrate pages proactively.

Multi-GPU Memory Management: In multi-GPU systems, page migration extends across GPUs. NVLink provides higher bandwidth for inter-GPU migration than PCIe. NVIDIA's multi-GPU memory management enables a single GPU kernel to transparently access memory on any GPU in the system, with the mapping and migration handled by the driver.

GPU virtual memory has transformed GPU programming from explicit, error-prone memory management to a more accessible model — enabling larger problems, simpler code, and transparent memory tiering across the heterogeneous memory hierarchy of modern computing systems.