Tensor Cores — specialized hardware units in NVIDIA GPUs that perform matrix-multiply-accumulate (MMA) operations at enormous throughput, designed to accelerate deep learning and HPC workloads.

What Tensor Cores Do

• Single operation: D = A × B + C, where A, B, C, D are small matrices (e.g., 4×4 or 8×4)
• One tensor core: Computes a 4×4×4 matrix multiply in a single cycle
• One SM has multiple tensor cores → massive parallel matrix throughput

Performance by Generation

Supported Precisions

• FP16, BF16: Standard training precision
• TF32: 19-bit format, drop-in for FP32 matrix ops (Ampere+)
• FP8 (E4M3, E5M2): Hopper+ for inference
• INT8, INT4: Quantized inference
• FP64: For HPC scientific computing (A100+)

How to Use

• PyTorch: torch.matmul() with torch.cuda.amp (automatic mixed precision) → tensor cores used automatically
• Requires specific matrix dimension alignment (multiples of 8 or 16)

Tensor cores deliver 10-20x higher throughput than standard CUDA cores for matrix operations — they're why GPUs dominate AI training.