unidirectional attention

Unidirectional attention restricts each token to attending only to previous tokens, enabling autoregressive generation. **Mechanism**: Causal mask applied, position i attends to positions 0 through i only. Same as GPT-style left-to-right attention. **Why unidirectional**: Generation is sequential, predict next token based only on past. Matches natural language production. **Training**: Can still train on full sequences using teacher forcing, but mask ensures each prediction uses only prior context. **Advantages**: Enables generation, simple and efficient, scales well. **Disadvantages**: Cannot use future context for understanding. Less effective for tasks requiring full sentence understanding. **KV cache benefit**: Since attention is only to past, key-value pairs can be cached and reused during generation. Huge speedup. **Used by**: GPT series, LLaMA, Claude, most production LLMs. **Comparison**: Bidirectional better for embeddings/understanding, unidirectional required for generation. Many approaches try to get benefits of both.