Mechanistic Interpretability is the subfield of AI safety and deep learning research that attempts to reverse-engineer neural networks by identifying the specific computations, circuits, and features implemented by individual neurons and attention heads — moving beyond "black box" explanations toward understanding what information is represented where and how it flows through the network, analogous to understanding computer programs by reading assembly code rather than just observing input-output behavior.

Core Goals

• Identify which neurons/attention heads detect which features (e.g., "token position", "gender", "syntactic subject")
• Trace information flow: Which components communicate with each other and why?
• Find circuits: Minimal subgraphs that implement specific behaviors (e.g., indirect object identification)
• Enable reliable safety claims: Understand whether a model can be trusted for specific tasks

Superposition Hypothesis

• Problem: Neural networks have more features to represent than neurons available.
• Solution: Networks encode features in superposition — multiple features per neuron, non-orthogonally.
• Evidence: Toy models with n features and d < n dimensions pack features at interference cost.
• Consequence: Single neurons are rarely monosemantic (one feature). They respond to many unrelated concepts.
• Implications: "Looking at activation of neuron 42" rarely tells you one clean thing.

Sparse Autoencoders (SAEs) for Interpretability

• SAE approach: Train sparse autoencoder on model's residual stream activations.
• Learn overcomplete dictionary: f(x) = ReLU(W_enc(x - b_dec) + b_enc)
• Reconstruction: x_hat = W_dec · f(x) + b_dec
• Sparsity penalty (L1): Forces each input to activate few features → monosemantic features emerge.
• Result: Dictionary features are often interpretable (e.g., one feature for "base64", one for "French words")
• Anthropic's findings: SAEs on Claude reveal thousands of interpretable features; some dangerous (e.g., "deception" features)

Attention Head Analysis

• Attention heads implement specific operations:
• Previous token head: Attends to immediately preceding token → implements recency.
• Duplicate token head: Attends to earlier occurrence of same token.
• Induction head: Matches [A][B]...[A] → predicts [B] → implements in-context learning.
• Induction heads are hypothesized to be the mechanistic basis for in-context learning.

Circuits: Indirect Object Identification (IOI)

• Task: "John gave Mary the book. She..." → Who is "she"? Mary.
• Wang et al. (2022) traced the circuit for this in GPT-2:
• S-inhibition heads: Find the subject (John).
• Induction heads: Detect repetition patterns.
• Name mover heads: Copy the indirect object (Mary) to final position.
• ~26 attention heads + MLP layers form the complete circuit.

Logit Lens / Residual Stream Analysis

• Residual stream: At each layer, model adds contribution to running sum.
• Logit lens: Unembed intermediate residual stream to token predictions → watch prediction evolve.
• Early layers: Often predict frequent tokens.
• Late layers: Refine to correct answer.
• Middle layers: "Recall" of stored knowledge.

Tools and Methods

Mechanistic interpretability is the field laying the scientific foundation for trustworthy AI — by moving from post-hoc explanations toward genuine understanding of what neural networks compute, mechanistic interpretability research aspires to give AI developers the tools to verify safety properties, debug unexpected behaviors, and make reliable claims about what a model is and is not capable of, transforming AI from an empirical art into an engineering discipline grounded in understanding.