Logit Lens › Background and Motivation

The logit lens was introduced by nostalgebraist in a 2020 blog post as a tool for examining GPT-2's internal computations. What motivated it was a basic puzzle: transformer language models produce strikingly capable outputs, but their internal processing is opaque. We can measure what the model outputs at the final layer, but we have no direct window into what it is "thinking" at intermediate layers.

Before the logit lens, the main tools for understanding intermediate representations were probing classifiers and dimensionality reduction techniques like PCA and t-SNE. These techniques require training additional models (probes) or involve lossy projections that discard most of the representational structure. The logit lens sidesteps these problems by using the model's own vocabulary-projection machinery as the decoder.

The key insight is that transformers have a specific, well-understood structure for converting hidden states to vocabulary predictions, and that structure consists of a layer normalization followed by a matrix multiplication. Its fixed parameters require no additional training. The question it answers, "what token would the model predict if it had to answer right now?", is interpretable and directly meaningful.

This technique connects two research traditions in mechanistic interpretability. The first asks about representations: what information is stored in the model's activations at each layer? The second asks about computations: what operations does the model perform to transform inputs to outputs? The logit lens leans toward the first tradition but illuminates the second by revealing when information becomes accessible in a form the model can act on.