Virtual Memory

Virtual memory is the technique by which an operating system and the processor's hardware give every running program its own private, uniform range of addresses, mapping those virtual addresses onto whatever physical RAM and disk space actually exists. A program sees what looks like a large, contiguous block of memory entirely its own; in truth the pieces may be scattered across physical memory, shared with other programs, or temporarily kept on storage, and the translation is done on every access without the program's involvement. The result is that programs are written against a clean abstraction instead of the messy, limited real memory of the machine.

The idea originated at the University of Manchester, where the Atlas computer of 1962 introduced paging and the notion of a one-level store. The goal was to let programmers ignore the split between small fast core memory and a large slow drum, with the hardware moving fixed-size blocks automatically. Peter Denning's working-set model of 1968 explained which pages a program needs to run well. Virtual memory then spread through mainframes and minicomputers and reached the x86 line with the Intel 80386 in 1985, which added paging hardware, making protected multitasking operating systems such as modern Unix-like systems and Windows NT practical on ordinary PCs. Today every general-purpose operating system uses it, including Linux, macOS, Windows, Android and iOS.

Mechanically, memory is divided into pages, typically 4 KiB on x86 and ARM, and a page table records where each virtual page lives. The memory management unit, a part of the processor, translates each virtual address by splitting it into a page number and an offset within the page, looking up the page number in the table, and substituting the physical frame number. Because walking a multi-level table for every access would be slow, a small cache called the translation lookaside buffer, TLB, remembers recent translations. If a page is not present in RAM the hardware raises a page fault, a kind of trap, and the operating system loads the page from disk, updates the table and restarts the instruction. On 64-bit x86 the virtual addresses in common use are 48 bits wide, a 256 TiB range per process, regardless of how much RAM is installed. Operating systems also support huge pages of 2 MiB or more to reduce translation overhead for large data.

Virtual memory delivers three benefits that go well beyond making memory look bigger. The first is isolation: each process has its own address space, so one program cannot read or corrupt another's data, and a stray pointer produces a controlled fault instead of a crash of the machine. The second is flexibility: the operating system can place pages anywhere, share a single copy of a library among many processes, and allocate memory lazily, giving physical pages only when a page is first touched. The third is extension: rarely used pages can be written out to swap space or a pagefile on a storage drive, letting the total demand exceed installed RAM, and files can be mapped into the address space, as with the mmap call, so reading memory reads the file. Modern techniques such as address space layout randomization also use it to make attacks harder.

The common misunderstandings concern the word virtual, the role of the disk and what the figures in a task manager mean. Virtual memory is not the same thing as swap space or as the paging file: those are only one part of it, and the mechanism is working even on a machine that never swaps. Total virtual size reported for a process counts address ranges that may never be backed by memory, so it can far exceed the physical RAM and say little about real usage. When a system really does need more pages than fit, it spends its time moving them back and forth, a collapse known as thrashing, and since storage is thousands of times slower than RAM the machine seems to freeze. On Linux, memory overcommit lets processes allocate more than exists, and when memory truly runs out the out-of-memory killer terminates a process.

For programmers and data scientists, virtual memory explains behaviors that seem puzzling. Libraries such as NumPy can memory-map a file with numpy.memmap so a dataset larger than RAM is read in pages as needed, which works well for sequential or sparse access and poorly for random access across a slow disk. GPU computing extended the idea: CUDA's Unified Memory, introduced with CUDA 6 in 2014, presents one address space across host and device and migrates pages on demand. Programs that touch memory in a scattered way suffer TLB misses, which is a reason that cache-friendly access patterns also matter here, and the random access memory and cache entries describe the layers that translation sits on top of.

Read virtual memory as a private, tidy map of addresses issued to every program, with the operating system and the memory management unit working behind the scenes to turn each imaginary location into real memory, and the cost that when reality runs short the illusion becomes very slow.