Mechanistic Interpretability › Summary

Mechanistic interpretability aims to reverse-engineer neural networks into human-understandable algorithms by identifying circuits, which are subgraphs of the model implementing specific computations.

Key concepts from this chapter:

• Circuits consist of nodes (attention heads, MLP neurons) connected by edges through the residual stream
• The QK circuit determines where a head attends; the OV circuit determines what it copies
• Activation patching identifies causally important components by restoring clean activations into corrupted forward passes
• Path patching identifies important edges, not just nodes, enabling precise circuit characterization
• Induction heads form a two-head circuit implementing pattern completion, discovered through careful circuit analysis, and represent a potential substrate for in-context learning
• Polysemanticity and superposition complicate circuit analysis by creating non-axis-aligned feature representations that do not map cleanly to individual neurons
• The linear representation hypothesis underpins much of mechanistic interpretability, though it has important caveats for nonlinear MLP computations

Mechanistic interpretability is complementary to the SAE-based feature interpretation approach from the previous chapter. Where SAE analysis decomposes representations into interpretable features, circuit analysis traces how those features flow and interact through computation. Together, they provide tools for understanding both what neural networks represent and how they use those representations to compute outputs. The long-term goal is a complete, verifiable account of model behavior at the computational level, a goal that currently remains out of reach for frontier-scale models but is advancing with each new methodological development.

The next chapter on activation steering uses mechanistic insights to go beyond understanding: by identifying the directions in activation space that encode specific concepts, we can intervene on model behavior by directly editing those directions during inference.