tensor core

```svg ```nsor core is a small matrix-multiply-accumulate engine built into each GPU streaming multiprocessor: in one instruction it multiplies two low-precision matrix tiles and adds them into a higher-precision accumulator — the operation deep learning spends most of its time on.\n\n**One instruction does a whole matmul.** A normal GPU (CUDA) core does one scalar multiply-add per cycle. A tensor core instead consumes two small matrix tiles — on the order of 16 x 16 — and produces D = A x B + C in a few cycles, so a single warp-level MMA instruction retires hundreds of multiply-accumulates. That is why a GPU's tensor-core FLOPS dwarf its general-purpose FLOPS.\n\n**Mixed precision is the trick.** The multiplies run in a low-precision format (FP16, BF16, and on newer parts FP8 or FP4) while the running sum accumulates in FP32. Low precision makes the multipliers small and fast and cuts memory traffic; the wide accumulator keeps the sum from losing accuracy. You get the speed of low precision with much of the stability of high precision.\n\n| GPU generation | Added formats | Notable |\n|---|---|---|\n| Volta (2017) | FP16 mul / FP32 acc | first tensor cores |\n| Turing (2018) | INT8, INT4 | inference |\n| Ampere (A100) | TF32, BF16 | structured sparsity |\n| Hopper (H100) | FP8 | Transformer Engine |\n| Blackwell (B200) | FP4 | 2nd-gen Transformer Engine |\n\n```svg\n\n```\n\n**Tensor core vs systolic array.** Both win the same way — massive operand reuse inside a matrix engine — but the shapes differ. A TPU is one large systolic grid fed by the compiler; a tensor core is a compact MMA unit replicated across every SM and invoked as a GPU instruction over register-file tiles. Nvidia's are the best known, but AMD's CDNA matrix cores and Intel's XMX engines do the same job.\n\nRead the tensor core through a quant lens rather than a marketing lens: a headline PFLOPS number only means something paired with its format (FP16 vs FP8 vs FP4) and whether it counts the 2:1 structured-sparsity mode. What decides real training and inference speed is achievable MACs per byte of HBM traffic at a tolerable precision — the tensor core raises the compute ceiling, but arithmetic intensity and accumulator width decide how much of it you actually reach.