Mechanistic Interpretability › Other Mechanistic Discoveries

The induction head is the most thoroughly characterized circuit, but mechanistic interpretability researchers have identified many other circuits and mechanisms. Each new discovery adds to a growing library of computational primitives that transformers use to process language.

Hanna et al. (2023) identified the circuit responsible for "greater-than" comparisons in GPT-2. When the model processes text like "The war lasted from 1943 to 19...", it correctly predicts digits that are greater than 43. The researchers identified a small set of attention heads and MLP neurons that implement this comparison.

The circuit works through a combination of positional encoding (which allows heads to detect relative positions of digits) and MLP neurons that implement threshold functions on numerical magnitude representations. The discovery is notable because it shows that something as abstract as numerical comparison is implemented by specific, identifiable components rather than being diffusely distributed across the network.

What makes the greater-than circuit particularly interesting is the role of the MLP layers. While the attention heads handle positional and contextual processing, the actual comparison logic lives in MLP neurons that function as threshold detectors. These neurons have input weights that respond to numerical magnitude and output weights that selectively promote or suppress the relevant digit tokens. This provides direct evidence that MLP layers actively implement logical operations rather than merely smoothing or transforming signals.

The circuit analysis also revealed why the model sometimes fails at greater-than comparisons. Failure modes trace back to specific circuit components: when the threshold neurons in the relevant MLP layers receive input from positions that lack clear magnitude information (as happens with ambiguous or unusual number formats), the comparison fails. This failure mode prediction is what distinguishes a mechanistic explanation from a merely correlational one.

Wang et al. (2022) at Anthropic performed a circuit analysis for indirect object identification (the "John and Mary" task described earlier). They identified 26 attention heads forming a circuit with distinct functional roles:

• Name mover heads: Copy the correct name (the indirect object) to the final position
• Backup name mover heads: Alternative pathways that activate when the primary movers fail
• Negative name mover heads: Suppress the incorrect name (the subject, not the object)
• S-inhibition heads: Inhibit the subject from being copied
• Induction heads: Participate in recognizing repeated tokens in the context
• Duplicate token heads: Identify which tokens appear multiple times (relevant to finding the non-subject name)

This multi-component circuit demonstrates that even a single linguistic task can decompose into a rich set of interacting mechanisms. The existence of backup pathways is particularly interesting: it suggests that transformers develop redundant implementations, which has implications for robustness and interpretability. When you ablate the primary name mover heads, model performance drops but does not collapse entirely, because the backup pathways partially compensate. This redundancy is not a design choice by the researchers; the model discovered it during training because gradient descent finds multiple parallel paths when they improve robustness.

The IOI circuit also provides a template for understanding how transformers handle co-reference. The model must decide which name in "When Mary and John went to the store, John gave a drink to" refers to the indirect object of the final clause. This requires integrating positional information (who appeared where in the sentence), semantic roles (who is the subject, who is the object), and token identity (which names are the same). The circuit analysis shows how attention heads with different specializations collectively solve this integration problem, with information flowing through the residual stream from early positional processing to later semantic-role disambiguation.

Algorithmic tasks are especially amenable to mechanistic analysis because they have precise, verifiable specifications. Researchers have studied how transformers implement:

• Modular arithmetic: Adding numbers modulo a prime, where Fourier representations appear in the weights
• Sorting algorithms: Simple sorting via attention-based comparison
• Bracket matching: Detecting balanced bracket sequences using running sum representations

The modular arithmetic work by Nanda et al. (2023) is particularly instructive. When a transformer learns to add numbers modulo a prime $p$, it discovers an unexpected algorithm: it represents numbers using Fourier modes (sine and cosine components at frequencies that are multiples of $2\pi/p$). The multiplication theorem for Fourier series then implements modular addition almost automatically. This result was unexpected: no one designed the network to use Fourier representations; the network found them because they are the natural solution to the problem.

The key observation is that modular addition satisfies a periodicity condition. For any prime $p$ and any integer $n$, we have $n \equiv n + p \pmod{p}$. This periodicity is naturally captured by Fourier modes: a representation using frequencies $k \cdot 2\pi/p$ for various integers $k$ automatically respects the modular structure. The network, through gradient descent, discovered this mathematical relationship and implemented it. The circuit analysis confirmed it by extracting the Fourier components from the model's weight matrices and verifying that they match the theoretically predicted frequencies.

This kind of discovery illustrates what makes mechanistic interpretability scientifically valuable. The goal extends beyond "can we explain this prediction" to "what general computational strategies do neural networks develop?" The Fourier-mode finding suggests that networks converge on principled mathematical solutions when those solutions are efficient, not on arbitrary numerical coincidences.

These "toy" algorithmic tasks are not merely academic. The representations found in them (Fourier modes, running counters, positional encodings) appear to be general computational primitives that also appear in language tasks. Understanding how the model implements a clean algorithm provides tools for recognizing similar structures in messier, more realistic settings.

The bracket matching work found that the model maintains a running sum in its residual stream: each token position carries information about the "balance" of the bracket sequence seen so far, much like a stack counter in a traditional parser. This representation appears not because the model was told to implement a stack, but because the stack is the optimal solution for tracking bracket balance. The mechanistic analysis verified this by showing that specific MLP neurons implement increment and decrement operations on this balance representation.

The progression from toy models to realistic applications follows a clear logic. In a toy model with a fully characterized circuit, you can verify the circuit is complete: every computation in the task is accounted for. This verified understanding then provides hypotheses to test in larger models. Circuits that appear in both toy models and realistic models are likely to reflect deep structural properties of transformer architectures rather than idiosyncrasies of specific training runs.