what is cpu cache
CPU cache is a small amount of extremely fast memory built directly into the processor to hide an enormous speed gap: a modern CPU can perform an operation in a fraction of a nanosecond, but fetching data from main system memory (RAM) can take a hundred times longer — so without cache, the fastest processor in the world would spend most of its time simply waiting for data to arrive.
```flowchart
{
"rows": [
{ "type": "nodes", "items": [
{ "title": "CPU core needs data", "sub": "check the fastest, closest place first", "tone": "neutral" }
]},
{ "type": "arrow" },
{ "type": "group", "title": "Cache hierarchy, checked in order", "items": [
{ "title": "L1 cache", "sub": "tiny, fastest, per-core", "tone": "green" },
{ "title": "L2 cache", "sub": "larger, still fast, per-core", "tone": "green" },
{ "title": "L3 cache", "sub": "largest, shared across cores", "tone": "orange" }
]},
{ "type": "arrow" },
{ "type": "nodes", "items": [
{ "title": "Main memory (RAM)", "sub": "last resort — far slower than any cache level", "tone": "blue" }
]}
]
}
```
**Cache works by keeping a small copy of recently and frequently used data much closer to the core than main memory ever could be.** Every time the CPU needs a piece of data, it checks the nearest cache level first; if the data is already there — a "cache hit" — it arrives almost instantly. If it isn't — a "cache miss" — the CPU has to check the next level out, and eventually main memory, with each step adding meaningfully more delay. Programs that repeatedly reuse the same data benefit enormously from cache, since after the first fetch that data usually stays close by.
```svg
```
**Each cache level trades capacity for speed, which is why there are several levels instead of one.** L1 cache is tiny — often just tens of kilobytes — but sits closest to the core and responds almost immediately; L2 is larger and slightly slower, usually still dedicated to a single core; L3 is far larger, shared across all the cores on the chip, and slower still, but remains dramatically faster than reaching out to main memory. Building one giant, uniformly fast cache would be prohibitively expensive and physically difficult to fit near the core, so the hierarchy spreads the tradeoff across several steps instead.
| Level | Typical size | Typical latency | Scope |
|---|---|---|---|
| L1 | Tens of KB | ~1 nanosecond | Private to one core |
| L2 | Hundreds of KB to a few MB | ~3-5 nanoseconds | Usually private to one core |
| L3 | Tens of MB | ~10-20 nanoseconds | Shared across all cores |
| Main memory (RAM) | Gigabytes | ~100 nanoseconds or more | Shared across the whole system |
**Cache size is a real, measurable factor in real-world performance, which is why chipmakers advertise it and sometimes sell nearly identical chips with different cache amounts as separate products.** A workload whose active data fits entirely inside a larger cache can run substantially faster than the same workload on a chip with a smaller cache, purely because it avoids the slower trip out to main memory more often — this effect shows up clearly in tasks like gaming and databases, where a chip with extra "stacked" cache can outperform an otherwise faster-clocked sibling chip.
**Cache effectiveness depends heavily on how predictable a program's memory access pattern is, not just on how much cache exists.** A program that repeatedly reuses a small, well-defined set of data — good "locality" — sees huge benefit from cache, since that data stays resident close to the core; a program that scans through enormous, scattered datasets with little reuse gains far less, because it keeps evicting old cached data to make room for new data it will likely never touch again.
Read CPU cache through a speed-versus-size pyramid lens: every step further from the core buys dramatically more storage capacity at the cost of dramatically more latency, and a processor's real-world speed on any given task depends heavily on how often it can answer a memory request from a fast, nearby level instead of making the long trip to main memory.