Graphics Processing Unit

A graphics processing unit is a processor built around thousands of small arithmetic units that execute the same operation on huge batches of data at once, originally designed to compute the color of millions of screen pixels sixty or more times per second and now equally central to scientific computing and artificial intelligence. Where a processor core is optimized to finish one instruction stream as quickly as possible, a GPU is optimized for throughput: the total amount of work completed per second across a very large number of independent tasks, even if each individual task is slower.

The GPU grew out of fixed-function graphics hardware, and the term itself was popularized by a product. Early 3D accelerators of the 1990s only handled parts of the rendering pipeline; Nvidia marketed the GeForce 256 in 1999 as the first GPU because it moved transform and lighting calculations onto the graphics chip. Programmable shaders arrived in the early 2000s, letting developers write small programs run per vertex and per pixel, and researchers soon noticed that a shader is just a function applied in parallel to a grid of numbers, which need not be an image. Nvidia released CUDA in 2007, a programming platform that let ordinary C-like code target the GPU directly, and the Khronos Group's OpenCL followed in 2009 as a vendor-neutral standard. That turned a graphics card into a general parallel computer.

Architecturally, a GPU trades the control machinery of a CPU for sheer arithmetic capacity. A CPU core spends most of its transistors on branch prediction, out-of-order logic and large caches to hide latency; a GPU instead groups simple lanes into units that run in lockstep, and hides latency by keeping thousands of threads in flight so that when one waits for memory another is ready to compute. Nvidia describes its model as SIMT, single instruction multiple threads, in which a group of threads, called a warp of 32 on its hardware, executes one instruction together. If threads in a warp take different sides of an if-statement, both sides run in turn with some lanes idle, so heavily branching code wastes the hardware. GPUs also carry their own very high-bandwidth memory, commonly called VRAM, built from GDDR chips or stacked high-bandwidth memory, separate from the main random access memory of the host computer.

CPU versus GPU: latency versus throughput same silicon budget, spent on different things CPU: a few strong cores Core 0 deep pipeline, branch predictor Core 1 out-of-order engine Large shared cache goal: finish one task with minimum delay good for branching, serial logic, operating systems GPU: thousands of simple lanes Groups of lanes executing one instruction in lockstep small caches, very high-bandwidth VRAM, many threads in flight goal: finish the most total work per second good for pixels, matrices, tensors, simulations

Deep learning is the workload that made the GPU one of the most strategically important chips in computing. Training a neural network consists overwhelmingly of matrix multiplications, which decompose into enormous numbers of independent multiply-and-add operations, exactly what a GPU does well. The turning point is usually dated to 2012, when Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton won the ImageNet competition with AlexNet, a convolutional network trained on a pair of Nvidia GTX 580 GPUs, by a wide margin over conventional methods. Nvidia then added dedicated matrix hardware, Tensor Cores, with the Volta generation in 2017, and the transformer architecture published that same year, which underlies modern large language models, is dominated by the matrix operations those units accelerate. Frameworks such as PyTorch and TensorFlow hide the device behind calls that move tensors to the GPU, and libraries such as cuDNN supply the optimized kernels. Rivals include AMD's ROCm software stack and special-purpose accelerators such as Google's TPU, first disclosed in 2016, which are not GPUs but compete for the same jobs.

Multiply two 1024 × 1024 matrices
about a billion multiply-adds, mostly independent
Copy matrices from main memory to VRAM
over PCIe, a transfer with real cost
Launch a kernel across many threads
Thread block 0
computes one tile of the result
Thread block 1
computes the next tile
Thread block N
all tiles run in parallel
Copy the result back, or keep it on the GPU
keeping data resident avoids repeated transfers

A GPU is not simply a faster processor, and misconceptions about that cost real time and money. Small problems, serial algorithms and code with unpredictable branches often run slower on a GPU than on a CPU, because launching work and moving data across the PCIe link can outweigh the arithmetic saved. Integrated GPUs share system memory with the processor and suit everyday display work, while discrete cards bring their own VRAM, and the amount of VRAM is frequently the practical limit on model size, which is why training and inference on large models pivot on memory capacity as much as on raw speed. Gaming still drives features like real-time ray tracing, which Nvidia put into dedicated RT cores in 2018. Precision matters too: neural networks tolerate 16-bit or lower number formats that scientific simulations cannot.

Read a graphics processing unit as a factory floor of thousands of simple workers all given the same instruction at once: spectacular when the job is a mountain of identical arithmetic, as in rendering and neural networks, and wasteful when the job is a single tangled chain of decisions.

Take graphics processing unit further

Ask the copilot about this term, or have our engineers assess it against your process.

Ask CFSGPT about this → Request a technical assessment → Open the CFS app