Mechanistic Interpretability › Superposition and Polysemanticity

A complication for mechanistic interpretability is that neural network representations are often not clean. The "one neuron, one concept" hypothesis is false in general. A single neuron frequently responds to multiple unrelated concepts, a phenomenon called polysemanticity.

The explanation for polysemanticity connects to the concept of superposition, which we covered in depth in the Sparse Autoencoders chapter. In brief: a layer with $d$ dimensions can represent more than $d$ distinct features if those features are represented as nearly-orthogonal vectors rather than axis-aligned directions. The model exploits the fact that most features are rare (they appear in only a fraction of tokens), so the interference between overlapping feature representations is usually small.

Superposition creates a fundamental challenge for mechanistic interpretability because the natural unit of analysis (the neuron or the attention head) does not align with the natural unit of meaning (the feature). A circuit for a specific task may be better described in terms of feature directions in the residual stream rather than individual neurons. When two unrelated concepts share a neuron, ablating that neuron disrupts both computations at once, making it difficult to attribute the effect to either concept specifically.

The Sparse Autoencoder approach (Chapter 8) attempts to recover the feature directions, giving a more interpretable basis for circuit analysis. When circuits are described in terms of SAE features rather than raw neurons, the explanations become cleaner and more meaningful. Instead of "this neuron activates for concept X," we can say "this SAE feature direction encodes concept X," and circuits can be expressed as operations over these feature directions. The integration of SAE-based feature analysis with circuit analysis is an active research direction, and several teams have begun mapping circuits in terms of SAE features rather than raw neurons.

The degree of superposition varies across the network. Early layers tend to have more superposition (they must represent many low-level features in a compact space), while later layers, which need to make finer distinctions, sometimes have sparser representations. MLP neurons also tend to be more polysemantic than attention heads, because MLP neurons interact with the residual stream through fixed weight vectors while attention heads can dynamically route information based on context.