what is a tpu
A TPU, or Tensor Processing Unit, is a type of AI accelerator chip designed specifically to speed up the tensor math — large-scale multi-dimensional array operations — that underlies neural network training and inference, built from the ground up for that specific computational pattern rather than adapted from general-purpose graphics hardware.
```flowchart
{
"rows": [
{ "type": "nodes", "items": [
{ "title": "Neural network math: massive tensor operations", "sub": "multi-dimensional array multiplication at huge scale", "tone": "neutral" }
]},
{ "type": "arrow" },
{ "type": "group", "title": "TPU built specifically for this pattern", "items": [
{ "title": "Purpose-built tensor math circuitry", "sub": "not adapted from graphics-rendering hardware", "tone": "green" }
]},
{ "type": "arrow" },
{ "type": "nodes", "items": [
{ "title": "High efficiency for AI training and inference", "sub": "at the cost of flexibility for non-AI workloads", "tone": "orange" }
]}
]
}
```
**A TPU is built from the ground up around tensor math, rather than being a graphics chip repurposed for AI.** GPUs were originally designed for rendering graphics and later found to be well-suited to AI workloads because both involve heavy parallel math; a TPU instead starts from tensor operations as the core design goal, arranging its circuitry specifically around the matrix multiplication patterns neural networks rely on — a purpose-built approach that can achieve higher efficiency for that specific math than hardware adapted from a different original purpose.
```svg
```
| Aspect | GPU | TPU |
|---|---|---|
| Original design purpose | Graphics rendering | Tensor/matrix math from the outset |
| Flexibility | Broad — handles many parallel workloads | Narrower — optimized specifically for tensor operations |
| Typical use | AI, graphics, general parallel compute | AI training and inference specifically |
| Availability | Widely available across many vendors | Often tied to a specific provider's hardware ecosystem |
**TPUs are typically deployed at large scale, linked together into pods to handle the enormous computational demands of training modern AI models.** A single TPU chip is only part of the picture — TPUs are usually designed to be connected together in large numbers with high-speed interconnects, forming a combined system that can train models far too large and computationally demanding for any single chip to handle within a reasonable timeframe.
**Because TPUs are purpose-built for tensor math, they can sacrifice the general-purpose flexibility a GPU retains.** A GPU can still be used effectively for graphics rendering, general parallel computing, or a wide range of other workloads beyond AI, whereas a TPU's design is narrowly optimized specifically around tensor operations — a tradeoff that yields greater efficiency for AI-specific workloads at the cost of being less useful outside that specific domain.
**TPU availability has often been tied closely to the specific provider that develops them, unlike GPUs which are broadly available from multiple vendors.** This means choosing to build on TPU hardware can involve a different set of tradeoffs around vendor dependence and ecosystem compatibility compared to choosing more broadly available GPU hardware, a consideration that factors into how organizations decide which AI accelerator hardware to standardize on for their workloads.
Read the TPU through a purpose-built lens: where a GPU brings AI-suitable parallel compute as a byproduct of its graphics-rendering origins, a TPU is designed from its very first principles around the tensor math neural networks depend on — a narrower focus that trades general-purpose flexibility for greater efficiency at the one job it was built to do.