activation patching

**Activation patching** is the **causal intervention method that replaces selected activations in one run with activations from another run to test influence on outputs** - it is one of the most widely used tools in mechanistic interpretability. **What Is Activation patching?** - **Definition**: Patch operation swaps activations at chosen layer, position, and component granularity. - **Purpose**: Measures whether a component carries task-relevant information for target behavior. - **Variants**: Can patch attention head outputs, MLP outputs, residual stream slices, or neuron groups. - **Readout**: Effect size is measured by changes in logits, probabilities, or task success metrics. **Why Activation patching Matters** - **Causal Evidence**: Directly tests necessity and sufficiency of internal signals. - **Circuit Discovery**: Helps isolate components that form behavior-driving pathways. - **Debugging**: Identifies where incorrect behavior first enters computation. - **Safety Analysis**: Useful for tracing risky output generation routes. - **Method Versatility**: Applies across many tasks and model architectures. **How It Is Used in Practice** - **Baseline Design**: Use paired clean and corrupted prompts with clear behavioral contrast. - **Granularity Sweep**: Start broad then narrow to specific heads or features. - **Robustness**: Repeat patch tests across multiple prompt templates to avoid spurious conclusions. Activation patching is **a foundational causal tool for transformer mechanism analysis** - activation patching is most reliable when experiment design cleanly isolates the behavior under study.