Attention is the Transformer operation that lets each token gather information from other tokens according to learned relevance rather than fixed distance.

Queries ask, keys match, and values carry content. Scaled dot-product attention is $\text{softmax}\!\left(\frac{QK^\top}{\sqrt{d_k}}\right)V$: query-key products create scores, softmax turns them into weights, and those weights mix the value vectors. The $\sqrt{d_k}$ factor keeps scores numerically well behaved as the head dimension grows.

Masking determines what information is visible. Encoders usually allow bidirectional attention, while autoregressive decoders use a causal mask so a token cannot see the future. Cross-attention uses queries from one sequence and keys and values from another.

The cost is memory as well as arithmetic. Standard attention forms a sequence-by-sequence score matrix, giving quadratic growth with context length. FlashAttention computes the same result in tiles to reduce memory traffic, while grouped-query and multi-query attention shrink the key-value cache used during inference.