Activation Patching › Connections to Other Interpretability Methods

Activation patching does not stand alone. It fits into a broader ecosystem of interpretability tools, each contributing a different type of evidence about how neural networks work.

Probing and activation patching are complementary. Probing tells you what information is accessible in a representation: whether a linear classifier can decode entity type, sentiment, syntactic role, or other features from a given layer's activations. Activation patching tells you what information is causally used for a specific output. The combination of high probe accuracy and high patch effect provides the strongest evidence that a feature is both present and used. Probe accuracy without patch effect suggests the information is present but bypassed by the model's actual computation. Patch effect without probing accuracy suggests the information is carried in a nonlinear or distributed form that the probe cannot detect.

Attention weight visualization, one of the earliest interpretability methods for transformers, reveals which positions each attention head attends to. Activation patching complements this by testing whether those attention patterns are causally important. An attention head might attend strongly to a particular position, but if patching that head's output has no effect on the behavior, the attention pattern is decorative rather than functional. Conversely, a head that attends weakly but consistently to a specific position might have a high patch effect because even weak attention carries important information. Activation patching provides the causal grounding that attention visualization alone cannot.

Activation patching is one of the core tools in the mechanistic interpretability toolkit, alongside attention pattern analysis, weight analysis, circuit verification, and sparse autoencoders. The typical workflow uses activation patching early in the analysis pipeline to identify candidate components, then transitions to other tools to understand what those components compute. The circuit discovered through activation patching provides a hypothesis to be explained mechanistically, and the mechanistic explanation in turn provides a more principled understanding of why the circuit works.

Building on this foundation, the Logit Lens chapter will show how to interpret what information the residual stream is encoding at each layer by projecting it directly into vocabulary space. While activation patching asks "where does information flow", the logit lens asks "what is the network predicting at each layer along the way". Together they form a powerful pair for understanding the temporal dynamics of information processing in transformers.