attention flow

**Attention Flow** is an **interpretability technique for transformer models that computes the effective attention by propagating attention weights across layers** — addressing the limitation that raw attention weights in a single layer don't capture the full information flow through a multi-layer transformer. **How Attention Flow Works** - **Attention Rollout**: Multiply attention matrices across layers: $A_{flow} = A_L cdot A_{L-1} cdots A_1$ (with residual). - **Residual Connection**: Account for skip connections by adding identity matrices: $hat{A}_l = 0.5 cdot A_l + 0.5 cdot I$. - **Attention Flow (Graph)**: Model attention as a flow network and compute max-flow from input to output tokens. - **Generic Attention**: Compute the "generic" attention as the flow through the attention graph. **Why It Matters** - **Multi-Layer Attribution**: Raw single-layer attention can be misleading — Attention Flow captures the complete information pathway. - **Token Attribution**: Shows which input tokens truly influence the output through all layers of the transformer. - **Visualization**: Produces heat maps showing the effective contribution of each input token to the prediction. **Attention Flow** is **tracing information through the transformer** — computing the effective end-to-end attention across all layers.