polysemantic neurons

**Polysemantic neurons** is the **neurons that respond to multiple unrelated features rather than a single interpretable concept** - they complicate simple one-neuron-one-concept interpretations of model internals. **What Is Polysemantic neurons?** - **Definition**: A single neuron may activate for distinct patterns across different contexts. - **Representation Implication**: Suggests compressed superposed coding in limited-dimensional spaces. - **Interpretability Challenge**: Feature overlap makes direct semantic labeling ambiguous. - **Evidence**: Observed through activation clustering and dictionary-based decomposition studies. **Why Polysemantic neurons Matters** - **Method Design**: Requires interpretability tools that go beyond single-neuron labels. - **Editing Risk**: Changing one neuron can unintentionally affect multiple behaviors. - **Compression Insight**: Polysemanticity reflects efficiency tradeoffs in representation capacity. - **Safety Relevance**: Hidden feature overlap can mask risky behavior pathways. - **Theory Development**: Motivates superposition and sparse-feature modeling frameworks. **How It Is Used in Practice** - **Feature Decomposition**: Use sparse autoencoders or dictionaries to split mixed neuron signals. - **Intervention Caution**: Avoid direct neuron edits without downstream behavior audits. - **Cross-Context Analysis**: Test activation meanings across diverse prompt domains. Polysemantic neurons is **a key phenomenon in understanding distributed transformer representations** - polysemantic neurons show why robust interpretability must focus on feature spaces, not only individual units.