Logit Lens › Interpreting Logit Lens Patterns

The logit lens reveals more than debugging information: it shows systematic patterns about how transformers process information. Researchers have identified several recurring patterns that appear across models and tasks.

The most common pattern is gradual refinement: early layers produce plausible but generic predictions, and later layers progressively sharpen toward the correct answer. This is consistent with the theoretical view of transformers as iterative refiners, where each layer builds on the previous one rather than performing independent computations.

The speed of convergence varies with difficulty. For "easy" predictions where the answer is clear from surface-level context (common collocations, syntactic completions like "I went to the" predicting "store"), the model converges quickly, often by layer 4 or 5 out of 12. For "hard" predictions requiring world knowledge or multi-step inference, convergence happens later and sometimes more abruptly.

The shape of the probability distribution can change dramatically from early to late layers, but raw logit-lens probabilities are not guaranteed to sharpen monotonically. In this GPT-2 Small example, an early layer is concentrated on a few generic continuations, while the final layer spreads probability across several plausible city names and ranks Paris first. That behavior illustrates both progressive semantic refinement and the raw lens's calibration limitations.

The early layer concentrates on generic continuations that fit the local phrase structure. By the final layer, the candidates become geographically specific and Paris ranks first, although the distribution remains broad. The contrast captures an important logit-lens lesson: semantic refinement does not necessarily mean a monotonic increase in confidence.

Across many prompts and models, the logit lens reveals that different types of information are committed at different layers. Research has consistently found that:

• Syntactic information (part of speech, subject-verb agreement, morphological form) tends to be incorporated early, in roughly the first third of layers (layers 1-4 in GPT-2 Small).
• Semantic information (word sense disambiguation, entity type, semantic role) is processed in middle layers (approximately layers 5-8).
• World knowledge (facts about named entities, geographic associations, historical events) tends to emerge in later layers (layers 8-12).

This layering of information types aligns with findings from probing classifier studies covered in the Probing Layers chapter. The logit lens provides a complementary, probe-free perspective: rather than asking what a trained probe can classify, we directly observe when the model's implicit prediction shifts to reflect each type of knowledge. The alignment is not perfect; individual attention heads and feed-forward layers can contribute knowledge at unexpected positions. But the broad pattern of syntactic-early, semantic-middle, factual-late is reliable enough to be practically useful.

Some prompts show a sudden jump pattern: the prediction is flat and uninformative across many layers, then abruptly changes to the correct answer in a single layer. This "phase transition" suggests that a specific computation at a specific layer is responsible for the answer, often associated with a particular attention head that routes a key fact from the position where it was mentioned to the prediction position.

These sudden jumps are of particular interest for mechanistic interpretability research, explored in the Mechanistic Interpretability chapter. When a single layer shows a large, discrete shift in the logit lens prediction, it is a strong signal that a circuit worth studying exists at that layer. Researchers have used logit lens "jump layers" as starting points for identifying induction heads and factual recall circuits.

The contrast between gradual refinement and sudden jumps also reveals differences in how the model represents different types of knowledge. Simple associations (collocations, syntactic patterns) tend to produce gradual refinement, while complex factual retrievals (capital cities, historical dates) often produce sudden jumps. This suggests the two types of processing are implemented differently in the network.

Transformers often appear to "know" the answer many layers before the final layer. The logit lens frequently shows the correct answer appearing as the top prediction by layer 8 or 9 out of 12 in GPT-2 Small, suggesting that the final three or four layers are doing refinement and confidence adjustment rather than introducing new information.

This raises an interesting question: what are the final few layers doing if the answer is already determined? Research suggests several functions. Final layers may be calibrating confidence by sharpening the probability distribution without changing which token is predicted. They may be resolving ambiguity between closely competing candidates. They may also be ensuring consistency with the broader discourse context in ways that are harder to measure than simple next-token prediction. The late layer effect is a reproducible observation, but its mechanistic explanation remains an active research question.

The heatmap view of the logit lens is particularly revealing because it shows how processing differs across token positions. Positions in the middle of a well-understood phrase converge quickly; positions at the boundary of ambiguous or knowledge-intensive structures converge slowly. A sentence like "Marie Curie was born in" will show rapid convergence for "Marie", "Curie", "was", and "born", because these are straightforward to predict in context, but slow convergence at "in", because the birthplace requires specific factual recall.

This position-dependent convergence pattern helps identify which parts of a text the model finds "easy" versus "hard." Positions where the logit lens converges slowly or shows oscillating predictions are exactly the positions where the model is doing its most complex processing, which makes them natural targets for deeper mechanistic analysis.