mechanistic interpretability

Mechanistic interpretability reverse-engineers neural network internals to understand the computations performed at the level of individual neurons, circuits, and features, aiming for scientific understanding of model behavior. Goals: (1) identify what features individual neurons detect (polysemanticity—neurons often represent multiple concepts), (2) map circuits (connected neurons implementing specific algorithms), (3) understand learned algorithms (how model solves tasks). Techniques: (1) activation patching (ablate/intervene to test causal role), (2) probing (train classifiers on activations to detect features), (3) circuit analysis (trace information flow through layers), (4) feature visualization (optimize inputs to maximize activations), (5) sparse autoencoders (decompose activations into interpretable features). Key findings: induction heads (copy patterns from earlier context), modular arithmetic circuits (grokking), and superposition (more features than dimensions through sparse encoding). Research centers: Anthropic, Redwood Research, EleutherAI. Relationship to AI safety: understanding how models work enables identifying failure modes, deceptive behaviors, and alignment issues. Challenges: scale (billions of parameters), superposition (features entangled), and polysemanticity. Comparison: behavioral interpretability (input-output analysis), mechanistic (internal computation analysis). Emerging field essential for building trustworthy and aligned AI systems through principled understanding rather than black-box testing.