Sparse Autoencoders: Dictionary Learning for LLM Features

Language models store an enormous amount of world knowledge in their weights, but the internal representations that emerge from training are notoriously opaque. A residual stream activation vector is just a dense list of floats, and a single neuron typically responds to dozens of unrelated concepts at once. This phenomenon, called polysemanticity, is the central obstacle to mechanistic interpretability: when every neuron means many things simultaneously, there is no clean mapping from model components to human-readable concepts.

Sparse autoencoders (SAEs) are a recently developed tool that tries to decompose these dense, entangled representations into a much larger set of sparse, interpretable features. The idea is rooted in a classical hypothesis from neuroscience and signal processing called the sparse coding hypothesis: that the brain (and by analogy, a trained neural network) represents information using a small number of active units drawn from a large dictionary. If that hypothesis holds, then we should be able to learn a large dictionary of features such that any given activation vector can be reconstructed as a sparse linear combination of a handful of dictionary elements.

The problem SAEs solve is more subtle than it first appears. You might ask: why not just look at what individual neurons do? The reason is that neurons in trained language models are almost never monosemantic. A single neuron in a large model might activate for "references to France", "the word 'also' used as a discourse marker", and "Python list comprehensions" all at once. This is not a training artifact that can be removed by better regularization. It reflects a fundamental geometric fact about how the model allocates representational capacity, and understanding it requires the kind of analysis SAEs are designed to provide.

Building on the interpretability tools covered in prior chapters, including activation patching and the logit lens, SAEs provide a complementary approach: rather than tracing causal pathways or projecting activations into vocabulary space, they decompose representations into a basis that aims to be both human-interpretable and computationally useful. In the next chapter on feature interpretation, we will see how those learned features are then labeled and used to build causal explanations of model behavior.

The motivation for sparse autoencoders comes from a long history in both computational neuroscience and signal processing. In 1996, Olshausen and Field showed that simple cells in the visual cortex can be understood as implementing a sparse code over natural images. When you train a dictionary on natural image patches and force sparse reconstruction, the learned atoms look like oriented Gabor wavelets, which is exactly what is found in primary visual cortex. The brain appears to use sparsity as an efficiency principle: by representing each stimulus with a small number of highly selective neurons, the brain minimizes metabolic cost while maximizing information capacity.

This insight from neuroscience is not coincidental. Efficient coding theory predicts that a system designed to transmit information under metabolic constraints will use a sparse code. If neurons are expensive to fire (and in the brain, they are), then the optimal strategy is to allocate firing to a small number of neurons that carry highly specific, non-redundant information. The rest remain silent. A neuron that fires for almost everything conveys little information per spike. A neuron that fires only when a very specific pattern is present conveys a lot.

The same insight has been applied to language model representations. Anthropic researchers, building on earlier work by Cunningham et al. (2023), demonstrated that the activation space of transformer language models can be decomposed into a large dictionary of sparse features, many of which are monosemantic: each feature fires for a coherent semantic concept (e.g., "names of fruits", "past tense verb forms", "code comments"). This result was surprising because the raw neuron activations showed severe polysemanticity, and it validated the core hypothesis that language models, like biological brains, may use sparse distributed codes.

The key intuition is that a model may have been trained with far more conceptual features in the world than it has neurons to represent them. The world has millions of meaningful concepts: countries, emotions, grammatical categories, coding patterns, historical events, and so on. A language model with tens of thousands of hidden units per layer cannot allocate a dedicated neuron to each concept. The network solves this problem by superposing many features across the same set of neurons, with each feature using a different linear direction in activation space. When features are superposed in this way, they look polysemantic when you examine individual neurons, but a sparse reconstruction algorithm can in principle separate them.

This is possible because high-dimensional spaces are roomy. In two dimensions, you can fit at most two perfectly orthogonal vectors. In a thousand-dimensional space, you can fit millions of vectors that are nearly orthogonal (with inner products close to zero). The language model exploits this geometric freedom by storing features as "almost orthogonal" directions, accepting small amounts of interference between features in exchange for enormous representational capacity.

Consider a concrete example of how superposition creates polysemanticity. Suppose a model has a 512-dimensional residual stream but needs to represent 10,000 semantic features. It cannot give each feature its own dimension. Instead, it assigns each feature a unit vector in that 512-dimensional space, chosen to be nearly orthogonal to all other features. When a feature fires, it adds its unit vector (scaled by the activation magnitude) to the residual stream. When multiple features fire simultaneously, their vectors add up. If no two features fire together very often (sparsity), the sum almost always contains just one or two terms, making it relatively easy to decode which features contributed. If features fire densely (nearly all the time), the sums are hopelessly entangled and decoding becomes impossible.

This is why sparsity is essential to the whole enterprise. The sparse coding hypothesis is not just "representations are sparse for efficiency." It is "representations must be sparse for the superposition to remain decodable." Without sparsity, a language model storing 10,000 features in 512 dimensions would produce activation vectors that are linear superpositions of hundreds of features simultaneously, and no algorithm could reliably disentangle them. Sparsity is what makes the geometry work.

The following visualization shows the core idea of superposition in two dimensions. Three features are packed into two-dimensional space by using directions that are not orthogonal. Each feature activates rarely (sparse), so the interference between them is low on average, even though they share the same space.

A sparse autoencoder is a two-layer neural network with a bottleneck-free design. Unlike a standard autoencoder that compresses input into a smaller latent space (which forces the model to discard information), an SAE expands input into a much larger latent space. The goal is to find an overcomplete dictionary of features: a set of direction vectors larger than the input dimension, such that any activation vector can be approximately reconstructed as a sparse combination.

The overcomplete design enables interpretability. If the dictionary were the same size as the activation space ($m = d$), we would recover a standard basis, and the decomposition would simply express activations in terms of that basis without any interpretability gain. If the dictionary were smaller ($m < d$), we would be compressing the representation and losing information, which is the wrong direction. By making the dictionary much larger ($m \gg d$), we give the SAE enough capacity to find a basis where each concept corresponds to a distinct dictionary element, even though this requires representing the data as a sparse combination of many potential atoms.

Think of this as the difference between a compact codebook and a rich vocabulary. A compressed representation uses a small set of basis vectors to cover all possible inputs densely. A sparse overcomplete representation uses an enormous vocabulary where each word (feature) is highly specific, so most words are silent for any given input. The advantage of the latter is that silence is informative: if a feature for "legal Latin phrases" does not fire, you know the input is not legal Latin. Each feature carries meaning both when it fires and when it does not.

Given an activation vector $\mathbf{x} \in \mathbb{R}^d$ taken from a specific layer and token position in a language model, the SAE computes a two-step transformation.

Encoder:

The encoder subtracts a pre-bias, applies a linear transformation, and then zeros out negative values with ReLU. This produces a sparse, non-negative feature activation vector $\mathbf{z}$:

where:

• $\mathbf{x} \in \mathbb{R}^d$: the activation vector from the language model (input to the SAE)
• $\mathbf{b}_{\text{pre}} \in \mathbb{R}^d$: a pre-encoder bias that centers the activations before encoding
• $\mathbf{W}_{\text{enc}} \in \mathbb{R}^{m \times d}$: the encoder weight matrix mapping from model space to feature space
• $\mathbf{b}_{\text{enc}} \in \mathbb{R}^m$: the encoder bias, one entry per feature
• $\mathbf{z} \in \mathbb{R}^m$: the sparse feature activations (most entries are zero because of ReLU)
• $m \gg d$: the dictionary size is much larger than the model's hidden dimension, making the representation overcomplete

The pre-encoder bias $\mathbf{b}_{\text{pre}}$ deserves special attention. Activation vectors in language models tend to have a non-zero mean: there is a "baseline" activation pattern that most tokens share, corresponding to the model's general processing state regardless of token content. By subtracting this baseline before encoding, the SAE focuses on the token-specific deviations from the mean, which is where the interesting semantic content lives. Adding it back during decoding ensures the reconstruction is in the original activation space. This centering step is a small but meaningful improvement over naive SAE designs that omit it.

The ReLU nonlinearity is what creates sparsity. Any pre-activation below zero is clamped to exactly zero. Features with small positive pre-activations also tend toward zero because the $L_1$ penalty (discussed in the next section) pushes them there. The result is a feature vector where most entries are exactly zero and a handful have positive values. The number of non-zero entries is the effective $L_0$ of the representation, which is the primary sparsity metric used in practice.

Decoder:

The decoder reconstructs the original activation vector from the sparse feature activations using a linear combination of the dictionary columns:

where:

• $\mathbf{W}_{\text{dec}} \in \mathbb{R}^{d \times m}$: the decoder weight matrix whose columns are the dictionary features
• $\hat{\mathbf{x}} \in \mathbb{R}^d$: the reconstructed activation vector
• The pre-bias $\mathbf{b}_{\text{pre}}$ is added back to re-center the reconstruction in the original activation space

The decoder columns $\mathbf{W}_{\text{dec}}[:, j]$ are the dictionary features: unit-norm vectors in the model's activation space. When feature $j$ is active (i.e., $z_j > 0$), its dictionary vector contributes to the reconstruction proportionally to $z_j$. The reconstruction is a sparse linear combination: at most a few of the $m$ features are active, so the sum has few non-zero terms. The remaining $m - k$ features contribute nothing.

Notice that the encoder and decoder are separate, untied matrices. This is an important design choice. In a tied autoencoder, $\mathbf{W}_{\text{enc}} = \mathbf{W}_{\text{dec}}^T$, which enforces a symmetry that is convenient mathematically but is not necessary and may hurt performance. Untied weights allow the encoder to learn a geometry optimized for detecting features (high response to the feature direction, low response otherwise) and the decoder to learn a geometry optimized for reconstruction (accurate linear combination of feature vectors). In practice, the rows of $\mathbf{W}_{\text{enc}}$ and the columns of $\mathbf{W}_{\text{dec}}$ end up nearly parallel to each other after training, because pointing the encoder row in the direction of the corresponding decoder column is the most efficient detection strategy. But the freedom to diverge is occasionally useful.

The ratio $m / d$ is called the expansion factor. A typical SAE might use an expansion factor of 4x to 64x or more. For a model with $d = 4096$ hidden units, an expansion factor of 8 yields $m = 32768$ features. The higher the expansion factor, the more features are available, but training becomes more expensive and individual features may become less coherent.

There is a meaningful tradeoff here. With a small dictionary, the SAE must pack many concepts into few features, recreating the polysemanticity problem it was meant to solve. With a very large dictionary, the SAE may split a single concept into fine-grained fragments (e.g., "fruit" splitting into "tropical fruit", "citrus fruit", and "stone fruit") or learn redundant features that respond to nearly identical contexts. Feature splitting is not always undesirable: if you want to understand how the model distinguishes between varieties of fruit, you need the fine-grained split. But if you want a high-level view of what categories the model tracks, you want features at a coarser granularity. In practice, researchers often train a range of SAE sizes and evaluate feature quality both qualitatively (by inspecting which tokens cause each feature to activate) and quantitatively (by measuring reconstruction quality).

The expansion factor also interacts with the sparsity level. If you have 16x more features than dimensions, you need each activation to use roughly $1/16$ of the features or fewer to maintain meaningful sparsity. Very high expansion factors require correspondingly tight sparsity constraints to avoid the SAE becoming a dense lookup table. The relationship between expansion factor and required sparsity is approximately:

where $k$ is the effective $L_0$ (number of active features). If $k = 10$ and $m = 160$ (expansion factor 10x for $d = 16$), the expected interference between active features is about 0.6, which is large enough to matter. If $m = 1600$, the interference drops to 0.06, making the superposition nearly negligible. This rough calculation explains why large expansion factors require tight sparsity: the more features you have, the fewer you can afford to use at once before interference degrades the representation.

A critical implementation detail is that the decoder columns are constrained to have unit norm after every gradient update. Without this constraint, the model could achieve sparse codes by making dictionary vectors very large and feature activations very small, avoiding sparsity in a trivial way. If the decoder column for feature $j$ has a large norm, then even a tiny activation $z_j = 0.001$ contributes substantially to the reconstruction, and the $L_1$ penalty on $z_j$ does not force it to zero because $0.001$ is already very small. The unit norm constraint removes this loophole by ensuring that achieving a strong reconstruction contribution for feature $j$ requires a large $z_j$, which the $L_1$ penalty can then force to zero.

After each parameter update step, we renormalize every decoder column to unit length:

where:

• $\mathbf{W}_{\text{dec}}[:, j]$: the $j$-th column of the decoder matrix, representing feature $j$'s direction in activation space
• $\|\mathbf{W}_{\text{dec}}[:, j]\|_2$: the Euclidean norm of that column
• The $\max(\cdot, 1)$ prevents norm collapse for dead features while renormalizing columns that have grown beyond unit norm

This renormalization is applied for every feature $j \in \{1, \ldots, m\}$ after each gradient step. The constraint also has an interpretive benefit: because all decoder columns have the same norm, the magnitude of feature activations is directly comparable across features. A feature with $z_j = 3.0$ is "three times as active" as a feature with $z_k = 1.0$, in a geometrically meaningful sense.

The SAE is trained to simultaneously minimize reconstruction error and encourage sparsity. The loss function combines a mean squared error (MSE) reconstruction term with an $L_1$ sparsity penalty on the feature activations.

For a single input activation vector $\mathbf{x}$, the loss is:

where:

• $\| \mathbf{x} - \hat{\mathbf{x}} \|_2^2 = \sum_{i=1}^{d} (x_i - \hat{x}_i)^2$: the squared Euclidean distance between the original activation and its reconstruction, penalizing inaccurate reconstructions
• $\| \mathbf{z} \|_1 = \sum_{j=1}^{m} |z_j| = \sum_{j=1}^{m} z_j$: the $L_1$ norm of the feature activations (all entries are non-negative after ReLU, so this equals the sum of activations)
• $\lambda > 0$: a hyperparameter that controls the tradeoff between reconstruction quality and sparsity; larger $\lambda$ forces sparser solutions at the cost of reconstruction accuracy

The reconstruction term pushes the SAE to be an accurate model of the activation space. The $L_1$ penalty pushes feature activations toward zero, making the representation sparse. This combination is identical to LASSO regression, a well-studied technique in statistics that simultaneously fits data and selects a small subset of predictors. The connection to LASSO is illuminating: LASSO is known to produce sparse solutions under conditions related to the restricted isometry property (RIP) of the design matrix. For SAEs, the analogous condition is that the true feature dictionary should have sufficiently spread-out, nearly-orthogonal columns, which is precisely the near-orthogonality assumption underlying the superposition hypothesis.

The natural measure of sparsity is the $L_0$ norm (count of non-zero elements), but $L_0$ minimization is NP-hard in general and not differentiable, so we cannot use standard gradient descent on it. The $L_1$ norm is the tightest convex relaxation of $L_0$ and has the desirable property of inducing exact zeros in solutions (unlike $L_2$ regularization, which only shrinks values toward zero without zeroing them out). To see why $L_1$ creates zeros while $L_2$ does not: the $L_2$ penalty has gradient $2z_j$ at any nonzero $z_j$, which pushes the activation toward zero but never quite reaches it because the penalty also shrinks to zero as $z_j \to 0$. The $L_1$ penalty has gradient 1 (for positive $z_j$), a constant pull toward zero regardless of the current activation magnitude. This constant pull is strong enough to zero out small activations entirely.

In practice, the ReLU nonlinearity in the encoder works in tandem with the $L_1$ penalty: the ReLU zeroes out small negative pre-activations, and the $L_1$ further suppresses small positive activations that would survive the ReLU. Together they create a two-stage filtering process: first, ReLU removes all negative pre-activations (roughly half, since pre-activations are centered around zero); second, $L_1$ training pressure removes small positive activations, leaving only the features with strong evidence from the encoder.

An important subtlety: even though the training loss uses $L_1$, researchers typically report the effective $L_0$ (actual count of non-zero activations) as the primary sparsity metric, because $L_0$ has a direct interpretation ("the number of features used per token") that $L_1$ does not. The two are correlated but not identical. A high-$\lambda$ SAE will have both a low $L_1$ and a low effective $L_0$, but the relationship between them is nonlinear: doubling $\lambda$ might halve $L_1$ while changing $L_0$ less dramatically if most of the reduction comes from shrinking already-small activations rather than eliminating more features.

Increasing $\lambda$ makes the SAE more sparse: fewer features activate per token, making each feature more selective. But higher sparsity means higher reconstruction error, since the SAE has fewer "building blocks" to explain each activation. This tradeoff is typically evaluated by plotting reconstruction quality against the average number of active features per token (called effective $L_0$, even though the training loss uses $L_1$).

A useful measure of reconstruction quality is the fraction of variance explained, sometimes written $R^2$ by analogy with regression. It measures how much of the variability in activation vectors the SAE accounts for:

where:

• $\| \mathbf{x} - \hat{\mathbf{x}} \|_2^2$: the reconstruction error (residual variance)
• $\| \mathbf{x} - \bar{\mathbf{x}} \|_2^2$: the total variance of the input activations around their mean $\bar{\mathbf{x}}$
• An $R^2$ of 1.0 means perfect reconstruction; an $R^2$ of 0 means the SAE does no better than predicting the mean

A well-trained SAE typically achieves $R^2 > 0.95$ while keeping the average number of active features (effective $L_0$) between 10 and 100. But $R^2$ alone is insufficient: it is possible to achieve high reconstruction quality with non-sparse features that are not interpretable. The joint optimization of high $R^2$ and low $L_0$ is what makes the features meaningful.

There is a second, more task-relevant quality metric called loss recovered. Rather than measuring how well the SAE reconstructs activations geometrically, loss recovered measures what happens when you substitute SAE-reconstructed activations back into the language model and run forward to compute the cross-entropy loss. If the SAE captures everything that matters for the model's predictions, the loss should barely change. If the SAE discards information needed for predictions in the pursuit of sparsity, the loss will increase. Loss recovered is typically defined as:

where $\mathcal{L}_{\text{SAE}}$ is the model loss with reconstructed activations, $\mathcal{L}_{\text{original}}$ is the baseline loss with original activations, and $\mathcal{L}_{\text{zero}}$ is the loss when all activations at that layer are zeroed out. A loss recovered of 1.0 means the SAE introduces no degradation. A value near 0 means the SAE is as damaging as removing the layer entirely. This metric is more meaningful than $R^2$ for practical interpretability work, because it measures whether the SAE captures the information the model uses.

The visualization below shows how reconstruction quality and sparsity vary with the $L_1$ coefficient $\lambda$. As $\lambda$ increases, the effective $L_0$ drops but $R^2$ also falls, showing the fundamental tradeoff.

Sparse autoencoders are closely related to the classical problem of dictionary learning in signal processing. Dictionary learning was originally developed for audio and image compression, where the goal is to represent signals as sparse combinations of basis atoms that are learned from data rather than defined analytically. The learned atoms capture the statistical regularities of the data domain: for natural images, these look like oriented edges; for music, they look like notes and chords.

Dictionary learning seeks a matrix $\mathbf{D} \in \mathbb{R}^{d \times m}$ (the dictionary) and sparse codes $\mathbf{Z} \in \mathbb{R}^{m \times n}$ (one column per data point) such that the reconstruction is accurate and each code is sparse.

Formally, the dictionary learning problem minimizes:

where:

• $\mathbf{X} \in \mathbb{R}^{d \times n}$: the matrix of $n$ data points (activation vectors stacked as columns)
• $\mathbf{D} \in \mathbb{R}^{d \times m}$: the overcomplete dictionary, with each column being a dictionary atom (a feature vector)
• $\mathbf{Z} \in \mathbb{R}^{m \times n}$: the sparse code matrix, one column per data point
• $\| \mathbf{Z}[:, i] \|_0 \leq k$: the sparsity constraint requiring at most $k$ non-zero entries per code vector
• $\| \cdot \|_F^2$: the squared Frobenius norm (sum of squared entries), measuring total reconstruction error

The classical approach alternates between a sparse coding step (fix $\mathbf{D}$, find sparse $\mathbf{Z}$ using matching pursuit or LASSO) and a dictionary update step (fix $\mathbf{Z}$, update $\mathbf{D}$ via K-SVD or gradient descent). SAEs reformulate this as a single end-to-end gradient descent problem: the encoder performs an approximate sparse coding step (using ReLU instead of an iterative solver), and the decoder columns are the learned dictionary.

The SAE approach sacrifices some theoretical optimality for practical scalability. Exact sparse coding via LASSO or matching pursuit on millions of activation vectors would be prohibitively expensive: each LASSO solve requires iterative optimization. The ReLU encoder provides an amortized approximation, learning to recognize which features are likely active from the input pattern alone, without iterating at inference time. A single forward pass gives a sparse code in $O(md)$ time, enabling training on the scale required for large language models.

It is worth understanding precisely what "amortization" means here. In classical dictionary learning, to find the sparse code for a new data point $\mathbf{x}$, you run an optimization procedure (e.g., basis pursuit or LASSO) from scratch for that specific input. This takes many iterations and scales poorly. An amortized approach trains a parametric function (the encoder) to predict the sparse code directly from the input. Once trained, the encoder runs in a single forward pass. The trade-off is that the encoder's approximation to the optimal sparse code is imperfect, particularly for unusual inputs that were not well-represented in the training distribution. This "amortization gap" is a known limitation of SAEs and motivates variants that run a few steps of optimization after the encoder's initial prediction.

The most common application of SAEs to language models is to train them on the residual stream activations at a specific layer. Given a trained language model, you collect a large dataset of text, run it through the model to record activation vectors at layer $\ell$, and then train an SAE on those vectors. The SAE is trained completely separately from the language model, which remains frozen throughout. This is important: the language model's weights are not updated, so the SAE learns to decompose existing representations rather than shaping how the model represents information.

This decoupled training strategy has both advantages and limitations. The advantage is that you can train many different SAEs (varying the hookpoint, dictionary size, or $\lambda$) without retraining the expensive base model. The limitation is that the SAE can only represent what is already in the activation space; it cannot discover features that the model has not already learned to encode.

SAEs can be trained on different components of a transformer. The choice matters because different components carry different kinds of information:

• Residual stream: The most common choice. The residual stream at each layer is the "main highway" of information, and SAEs trained there tend to learn the cleanest features. Because the residual stream aggregates contributions from all attention heads and MLP layers up to that point, features at later residual stream positions reflect the cumulative computations of all preceding layers.
• MLP output: The MLP sublayer is where most knowledge storage is thought to occur. SAEs trained on MLP outputs often learn factual and entity-level features, including features that correspond to specific persons, places, and concepts from the training data.
• Attention output: SAEs trained on attention output layers learn features related to syntactic relationships and positional information, such as subject-verb agreement patterns and co-reference.
• Attention query/key: Less common, these can reveal patterns in how attention heads select information. Analysis here may help understand which features cause an attention head to attend to a particular token.

The choice of hookpoint affects both what features are learned and how they can be used for downstream analysis. Residual stream SAEs are the most general: because the residual stream aggregates contributions from all earlier layers, features there can reflect both syntactic and semantic information simultaneously. MLP SAEs tend to be more focused on factual knowledge and less on syntactic structure, which can be an advantage when you want to study how the model stores and retrieves facts.

Different layers also yield different types of features. Earlier layers tend to encode lower-level syntactic properties (part of speech, morphology, local context), while later layers encode higher-level semantic properties (entity type, argument structure, discourse relations). This layer-by-layer progression mirrors what has been found using probing classifiers. It also suggests a natural experimental strategy: if you want to understand when a particular concept is computed, you can train SAEs at many layers and observe at which layer a feature for that concept first appears with high fidelity.

Training high-quality SAEs requires enormous amounts of data. The SAE is trained on a distribution of activation vectors, and every forward pass through the language model generates one activation per layer per token. A typical training run for a large-scale SAE uses billions of tokens. For a model with $d = 4096$ and a dictionary size of $m = 16384$, this means processing on the order of $10^{12}$ floating-point numbers across the training run.

Fitting the SAE weights alone does not explain the scale requirement. The SAE has only $md + d + m$ parameters (encoder weights, pre-bias, and encoder bias), which for $d = 4096$ and $m = 16384$ amounts to roughly 67 million parameters. You do not need billions of examples to fit 67 million parameters in a traditional sense. The need for scale comes from feature coverage: rare features, such as a feature for a specific historical event or a niche programming pattern, might activate on only a tiny fraction of tokens. If the training dataset does not contain enough examples of that pattern, the SAE may fail to learn a coherent feature for it. The SAE needs to see each feature activate enough times (perhaps hundreds of times at minimum) for the gradient signal to shape the corresponding dictionary vector reliably.

For practical intuition: if feature $j$ fires on $0.001\%$ of tokens (one per 100,000), you need at least 100 million tokens to see that feature fire 1,000 times. For features with activation rates of $0.0001\%$ or lower, you need 10 billion or more tokens. Real language models have millions of features at varying activation rates, and ensuring adequate coverage of even moderately rare features is the primary driver of the data scale requirement.

Anthropic's publicly released SAEs for Claude's Sonnet model, for example, were trained with hundreds of billions of activation samples and contain millions of features. Google DeepMind's Gemma Scope project released SAEs for the Gemma 2 model family with similar scale. These large-scale training efforts are expensive but produce features with high fidelity and semantic coherence.

Once an SAE is trained, how do you know if a feature is meaningful? The standard approach is max-activating example analysis: for each feature $j$, collect the top-k tokens (or token contexts) that produce the highest activation $z_j$. If those contexts share a coherent semantic theme (e.g., the top-activating contexts for a feature are all city names), the feature is considered interpretable.

This approach is analogous to the receptive field analysis used in neuroscience: to understand what a neuron encodes, you find the stimuli that drive it most strongly. A neuron in visual cortex that responds maximally to vertical edges oriented at 45 degrees is interpretable; a neuron that responds to a mix of unrelated visual patterns is not.

This approach is inherently subjective, so researchers have also developed automated interpretability methods. These typically use a language model to read the top-activating contexts for a feature and generate a natural language description (e.g., "This feature activates for mathematical formulas in LaTeX"), then verify that description by checking whether the feature activates for new contexts that match the description and does not activate for contexts that do not. The verification step is what distinguishes a good feature label from a spurious coincidence in the top-activating examples.

Beyond max-activating examples, researchers also examine the feature's effect on model outputs by activation steering: artificially increasing or decreasing $z_j$ and observing how the model's predictions change. If boosting a "feature for Paris" feature increases the probability of tokens like "France", "Eiffel Tower", and "Seine", that is strong evidence the feature encodes a concept related to Paris. If boosting a "feature for Paris" has no coherent effect on outputs, the feature label may be wrong or the feature may not be causally relevant to the model's computations.

Large-scale SAE experiments on frontier language models have produced a remarkable catalog of learned features. The sheer variety of what these features represent is itself an important empirical finding.

Some features are highly specific to surface-level token patterns. There are features that activate for text in specific scripts (Arabic, Cyrillic, Korean), features that fire for specific programming language keywords, and features that respond to punctuation in specific positions. These are relatively unsurprising: surface-level regularities are easy to learn and are presumably useful to many downstream computations.

More striking are features that appear to encode abstract semantic concepts. Anthropic's analysis of Claude models identified features for specific named entities, where a single feature would activate for all mentions of a particular person across different contexts, names, and phrasings. A feature for "Albert Einstein" might activate for "Einstein", "the physicist who developed relativity", "E = mc^2", and "the author of the special theory of relativity". The feature is not tracking a surface pattern (these are very different strings) but an abstract identity.

There are also features that track relational and structural properties of text. Features for "beginning of a list", "the token following a colon", "indirect speech", and "the subject position in a clause" have all been identified. These structural features illuminate how the model's internal representations encode content along with the grammatical and discourse structure of the text it processes.

Perhaps most intriguing are features that appear to encode emotional and intentional states. Features activating for "sycophantic language", "the speaker expressing uncertainty", and "text from an AI assistant responding to a human" have been found. The latter category points toward a broader use of SAEs: understanding what the model knows and how it represents context, roles, and interaction patterns. This is potentially relevant to AI safety research, which aims to understand how models represent goals, intentions, and the relationship between themselves and their users.

To build intuition, let's work through how an SAE would learn to decompose a small synthetic dataset of activations. Imagine we have a 2-dimensional activation space, but the actual generative factors are 4 binary features: $f_1$ (indicates a verb), $f_2$ (indicates a proper noun), $f_3$ (indicates negation), $f_4$ (indicates past tense). Each activation vector is a linear combination of whichever features are active, with some noise added.

The SAE should learn to recover these 4 underlying features from the 2D activations, even though the problem is technically underdetermined (we have more features than dimensions). This is possible because of sparsity: at most one or two features are active at a time, which makes the problem well-conditioned. Intuitively, if you observe a large collection of 2D vectors and know that each was generated by at most 2 of 4 possible patterns, the geometry of the data (clustering near the sparse combinations) reveals the underlying patterns.

Let us trace through this concretely. Suppose feature $f_1$ (verb) corresponds to the direction $(1, 0)$ and feature $f_2$ (proper noun) corresponds to the direction $(-0.5, 0.87)$ in 2D space. When we observe an activation vector near $(0.9, 0.1)$, sparsity tells us it is most likely a single-feature activation for $f_1$. When we observe a vector near $(-0.4, 0.75)$, it is likely a single-feature activation for $f_2$. When we observe a vector near $(0.5, 0.9)$, it could be a two-feature combination of $f_1$ and $f_2$. The SAE encoder learns to map each of these observations to the correct sparse code by projecting the input onto the feature directions and thresholding.

The key insight is that you cannot determine this decomposition from any single observation; you need the statistics of many observations. The SAE essentially learns that the data is concentrated near a small set of directions, with some observations at intermediate points from sparse combinations. This geometric structure is what the ReLU encoder exploits. Directions with high dot product against the input are likely active features, and the decoder learns to reconstruct the input as a combination of those directions.

In practice, the dictionary directions for the 4 features must be nearly orthogonal to allow unique decoding. With 4 features in 2D, they cannot be fully orthogonal (orthogonality requires no more features than dimensions), but the near-orthogonality of high-dimensional space makes this tractable at the scale of real language models. A 4096-dimensional residual stream can comfortably support tens of thousands of nearly orthogonal features, and the near-orthogonality is what ensures each feature can be decoded independently.

This also explains why the SAE approach breaks down in very low-dimensional spaces. The 2D example above is illustrative but not realistic: with only 2 dimensions and 4 features, the features are forced to interfere substantially. As dimension grows, the interference per feature pair decreases as $1/d$, making the decomposition increasingly reliable. Language models, with hidden dimensions in the thousands, operate in a regime where the interference is small enough to make SAE decompositions informative.

Let's implement a minimal SAE and train it on synthetic activation data that simulates the superposition of independent features. This implementation follows the design used in Anthropic's open-source SAE research, simplified for clarity.

We generate 50,000 synthetic activation vectors where each is a superposition of a small number of the 64 true features, compressed into a 16-dimensional space. This simulates the superposition hypothesis at small scale: more features than dimensions, with each token using only a few of those features at once. Each true feature has a probability of 0.05 of being active on any given token, so on average about 3.2 features are active simultaneously (roughly 5% of 64). This is deliberately a challenging setting: the 64 features are packed into 16 dimensions, and the interference between simultaneously active features creates overlapping activation patterns.

The expansion factor of 8x gives us 128 dictionary features to explain patterns in 16-dimensional activation space. Notice that the encoder and decoder are separate matrices (not tied weights), which allows the encoder to learn a different geometry for feature detection than the decoder uses for reconstruction. The encoder rows are optimized to respond strongly to their target feature direction and weakly to others, while the decoder columns are optimized to reconstruct activations accurately when linearly combined. In real language models, expansion factors of 4x to 64x are common; larger factors trade computational cost for the ability to represent more fine-grained features.

After training, it is important to check that the learned features are used. A common failure mode is feature collapse: some features become very popular (active for most tokens) while others are never activated at all. The distribution of feature activation frequencies tells you how well the dictionary is utilized.

A significant practical challenge in training SAEs is the dead features problem. When a feature never activates, its gradient from the $L_1$ term is zero (since $\partial |z_j| / \partial z_j = \text{sign}(z_j)$ is undefined at zero), and the reconstruction gradient for that feature's decoder column is also zero (since the feature contributes nothing to the output). The feature is stuck and cannot recover without external intervention. This is a form of the "neuron death" problem familiar from training deep networks with ReLU activations.

To understand why a feature becomes dead in the first place, consider what happens early in training. The encoder weights are randomly initialized, so some encoder rows initially point in directions that are nearly orthogonal to all data points. These features produce very small pre-activations, which the ReLU sets to zero. The $L_1$ gradient has nothing to push (the activation is already zero), and the reconstruction gradient also vanishes (the feature contributes nothing to the output). The feature is in a stable zero state from the start and receives no learning signal.

Dead features are especially problematic because they waste dictionary capacity. If 20% of your features are dead, you are effectively operating with an 80% smaller dictionary. Worse, the remaining features must cover the full diversity of activation patterns, potentially becoming less specialized. In extreme cases, a few highly active features attempt to cover the entire activation space, leading to polysemantic features that undermine the whole purpose of the SAE.

Several solutions have been proposed to address this problem:

Neuron resampling (Anthropic, 2023): Periodically check which features have not activated in a recent window (e.g., the last 100,000 training steps). For each dead feature, reinitialize its encoder row to point toward a high-reconstruction-error data point, scaled by the average norm of living encoder rows. This gives the dead feature a "kick" toward a useful region of activation space where it can start receiving gradient signal. The resampling is repeated periodically throughout training.

Auxiliary loss (EleutherAI, 2024): Add an auxiliary reconstruction loss using the top-$k$ dead features as a "dead feature autoencoder". Specifically, take the residual $\mathbf{x} - \hat{\mathbf{x}}$ from the main SAE and train the dead features to reconstruct it. This directly trains dead features to reconstruct patterns that the main SAE is currently failing to represent. The auxiliary loss is scaled by a small weight so it does not dominate the main objective.

TopK activation (Anthropic, 2024): Replace the ReLU encoder with a TopK activation function that always keeps exactly $k$ features active. This guarantees a constant $L_0$ and eliminates the dead feature problem, since every feature is always used for some fraction of data points (as long as the initialization is reasonable). The tradeoff is that the effective sparsity is fixed rather than learned, removing one degree of freedom from the model.

Beyond the basic architecture, several variants have been developed to address specific limitations or explore different training regimes. The field is evolving rapidly, and the choices made by major research groups in their production SAEs reflect lessons learned from large-scale experiments.

The standard ReLU activation creates a discontinuity in the gradient at zero, which makes training noisy. More subtly, the ReLU means that features with pre-activations just above zero are treated the same as features with large pre-activations, as long as both survive the threshold. JumpReLU SAEs (Rajamanoharan et al., 2024) replace the ReLU with a learned threshold function. For feature $j$, the activation is:

where:

• $h_j = (\mathbf{W}_{\text{enc}} \mathbf{x})_j + b_{\text{enc}, j}$: the pre-activation for feature $j$ (the raw encoder output before thresholding)
• $\theta_j \in \mathbb{R}$: a learned per-feature threshold, optimized alongside the encoder and decoder weights using a straight-through estimator for the gradient of the step function

The key advantage is that the threshold $\theta_j$ can adapt per feature, allowing some features to be very selective (high threshold, activates only for strong examples) while others are more broadly active (low threshold, activates for weak instances). The sparsity penalty is also modified: rather than penalizing the magnitude of activations, it penalizes the binary activation indicator, which better reflects the true $L_0$ objective.

The gradient challenge is real: the step function in the JumpReLU is not differentiable with respect to $\theta_j$. The solution is the straight-through estimator: during the forward pass, the threshold is applied exactly; during the backward pass, the gradient of the step function is replaced by a smooth approximation (such as a Gaussian kernel around $\theta_j$). This allows gradient-based optimization to adjust the threshold while using the exact threshold during inference.

Standard SAEs are trained with a fixed dictionary size, and changing the size requires retraining from scratch. Matryoshka SAEs (named after Russian nesting dolls) use a nested training objective where features are organized into groups of increasing size, and each prefix of features is trained to be a complete SAE by itself. This produces a hierarchy where the first $k$ features provide a coarse representation, and adding more features refines it.

The practical benefit is flexibility: a single Matryoshka SAE can be evaluated at multiple sparsity levels. For interactive analysis where you want to start with a coarse view and drill down, this is much more efficient than training separate SAEs for each sparsity level. The nesting property also enables efficient multi-granularity analysis: you can ask "what are the 10 most important features for this activation?" and "what are the 100 most important features?" using the same trained model.

Rather than training a single SAE on raw activations, residual SAEs train a sequence of SAEs. The first SAE reconstructs the activation as well as possible. The second SAE is then trained to reconstruct the residual (error) of the first SAE. The third SAE reconstructs the residual of the second, and so on. This cascade can achieve better reconstruction quality at a given total dictionary size than a single large SAE, because each stage focuses on patterns the previous stages could not capture.

Residual SAEs draw inspiration from boosting in ensemble methods: each new learner focuses on the mistakes of the previous learners. The key difference is that in residual SAEs, the stages are SAEs with sparse codes, so the overall representation remains interpretable as a sparse combination of features from multiple stages. One practical benefit is that the first-stage features tend to capture the most dominant and common patterns, while later-stage features capture subtler, rarer patterns. This natural hierarchical organization can be useful for interpretability analysis at different levels of granularity.

Evaluating SAEs is an active research area because there is no single ground-truth measure of feature quality. Researchers use a combination of quantitative and qualitative metrics, and the best-performing SAEs by one metric are not always the best by others. This makes SAE evaluation difficult.

The core quantitative tradeoff is between reconstruction fidelity and sparsity:

• Variance explained ($R^2$): What fraction of the variance in activation vectors is explained by the SAE? Higher is better. This is the most commonly reported metric and is easy to compute, but it does not distinguish between faithful geometric reconstruction and reconstruction of task-relevant information.
• Mean $L_0$: The average number of features active per token. Lower means sparser. Combined with $R^2$, this defines the Pareto frontier used to compare SAEs at the same sparsity level.
• Loss recovered: When you replace language model activations with SAE reconstructions and run the model forward, how much does the language model's loss increase? A loss recovered of 100% means no degradation. This metric is more meaningful than $R^2$ for practical interpretability work because it captures whether the SAE preserves the information the model uses for predictions, rather than variance that may be irrelevant noise.
• CE loss difference: The absolute increase in cross-entropy loss when the model uses SAE-reconstructed activations. Smaller is better. A model with good $R^2$ but high CE loss difference is geometrically accurate but discarding task-relevant information.

A useful summary is the Pareto frontier of reconstruction quality vs. sparsity across different values of $\lambda$. A better SAE traces a frontier further from the origin: for any given sparsity level, it achieves better reconstruction.

Beyond numerical metrics, researchers evaluate SAEs qualitatively. These evaluations require human judgment or proxy language models:

• Monosemanticity rate: What fraction of features, when inspected, have a clear, coherent semantic interpretation? This is typically measured by having human annotators or automated language model labelers review the top-activating examples for each feature.
• Feature coherence: Do the top-activating examples for a feature share a specific semantic property, or do they span unrelated concepts? A feature is coherent if you could describe it with a single short phrase and have someone use that description to predict (with above-chance accuracy) whether a new example would activate the feature.
• Recall of known circuits: When you use the SAE to analyze known circuits (e.g., induction heads, factual recall mechanisms), do the SAE features correspond to the components identified by other interpretability methods? This is a cross-validation of the SAE against independent evidence.
• Activation steering consistency: When a feature is artificially activated via steering, do the resulting model outputs match the expected semantic content of the feature? A feature labeled "Paris" should, when steered up, increase the probability of Paris-related tokens.

The last two metrics are particularly important because they connect the SAE's features to causal claims about model behavior. A feature might be statistically coherent (high monosemanticity rate) but causally inert: it activates for a concept but does not causally influence what the model outputs. For interpretability research, causal relevance is what ultimately matters.

Sparse autoencoders are a powerful tool, but they come with several important limitations that constrain how we should interpret their outputs.

Faithfulness vs. completeness: An SAE can reconstruct activations accurately without the learned features being the actual computational primitives the model uses. The SAE optimizes reconstruction loss, not interpretability. A feature that fires for "city names" might not correspond to any discrete circuit in the model; it could be an artifact of how the SAE decomposes a continuous representation. Reconstructing an activation well is necessary but not sufficient for the features to be causally meaningful. This is perhaps the deepest limitation: the SAE gives you a decomposition, but it cannot guarantee that decomposition reflects the model's internal organization rather than an alternative basis that is statistically equivalent for reconstruction purposes.

The basis non-uniqueness problem: If the true features in a language model's representations form a nearly orthogonal dictionary, then the SAE will reliably recover them. But if the model's representations are smooth manifolds or continuous distributions rather than discrete, sparse features, then any overcomplete dictionary that spans the manifold will achieve good reconstruction, and the specific features found depend on the initialization, the $\lambda$ value, and the training order rather than on anything fundamental about the model. Different SAEs trained on the same activations may find different feature dictionaries that are equally good by reconstruction metrics but interpret the data differently.

Feature splitting and merging: As the dictionary size increases, a single concept may be split across multiple features (e.g., "fruit" splits into "tropical fruit", "citrus fruit", "stone fruit"). Conversely, small dictionaries may merge related but distinct concepts. There is no canonical granularity for features, and the right granularity likely depends on the downstream application. Consequently, comparing features across SAEs of different sizes requires care, and "feature counts" are not a stable measure of representational complexity.

Training instability: SAE training is sensitive to the $\lambda$ hyperparameter, the learning rate, and the decoder normalization schedule. Small changes can cause feature collapse, dead features, or poor reconstruction. Practitioners often train multiple SAEs with different hyperparameters and select the best based on the Pareto frontier of reconstruction quality vs. sparsity. The sensitivity is partly inherent to the $L_1$-regularized optimization field, which has many local minima, and partly due to the interaction between the dead feature problem and the decoder normalization.

Computational cost: Training SAEs on large models requires collecting billions of activation vectors. For frontier language models with thousands of layers, training SAEs for every layer and hookpoint requires enormous computational resources. The Gemma Scope project, for example, used significant Google TPU time to train SAEs across all layers of the Gemma 2 model family. This scale requirement means that analysis of SAEs across all layers and hookpoints of frontier models is accessible primarily to large research organizations, limiting independent verification of results.

The superposition hypothesis may be wrong: SAEs are motivated by the assumption that model representations are superpositions of sparse, monosemantic features. But this is a hypothesis, not an established fact. If models use distributed representations that are not sparse in any basis, SAEs would learn features that are artifacts of the decomposition rather than computational building blocks. The striking results from large-scale SAE analysis are suggestive but not conclusive, because they are difficult to verify independently. We know that SAEs produce interpretable features, but we do not know whether those features constitute a complete and non-redundant description of what the model is computing.

The labeling problem: Even when SAE features are coherent, labeling them at scale is expensive. A million-feature SAE requires a million feature labels. Automated labeling using language models can provide labels quickly but at lower quality than human annotation, and the quality is difficult to verify systematically. Errors in feature labels can propagate to downstream analyses that treat those labels as ground truth.

Despite these limitations, SAEs have produced striking results. Anthropic's analysis of Claude Sonnet's SAE features identified millions of features with coherent semantics, including features for famous individuals, programming concepts, emotional states, and many more. The feature for the name "Michael Jordan" activated across many distinct contexts, from basketball to statistics to music, suggesting the SAE had found a representation encoding the concept of a specific person rather than just a surface-level token pattern. These results demonstrate that SAEs are revealing something real about language model representations, even if the full theoretical picture of why they work so well remains incomplete.

Successful SAE training requires careful attention to several key parameters. The choices interact with each other, so understanding the tradeoffs is important:

• d_sae (dictionary size): The number of features in the dictionary. Larger dictionaries find more fine-grained features at greater computational cost. Typical expansion factors range from 4x to 64x. Too small risks polysemantic features; too large risks feature splitting and increased dead features.
• lambda_l1 (sparsity coefficient): Controls the tradeoff between reconstruction quality and sparsity. Larger values produce sparser but less accurate reconstructions. Must be tuned carefully; typical values range from $10^{-4}$ to $10^{-1}$. The right value depends on the expansion factor and the intrinsic sparsity of the activation distribution.
• hookpoint: Which component of the language model to extract activations from. Residual stream activations are the most common choice for general-purpose interpretability. MLP outputs work better for studying factual knowledge storage.
• n_training_tokens: The number of token activations to train on. High-quality SAEs typically require billions of tokens to cover rare features. The requirement scales with the rarity of the rarest features you want to capture.
• activation function: ReLU with $L_1$ penalty, TopK, or JumpReLU. TopK is increasingly preferred for its stability and elimination of the dead feature problem. JumpReLU offers per-feature threshold learning at some additional complexity.
• decoder normalization: Always normalize decoder columns to unit norm after each gradient step. Omitting this allows the model to trivially satisfy the sparsity constraint without learning sparse representations.

Sparse autoencoders address the polysemanticity problem in language models by decomposing dense, entangled activation vectors into sparse combinations of interpretable features. The architecture is simple: an overcomplete encoder that maps activations to a larger space via ReLU, and a linear decoder with unit-norm columns. Training optimizes a reconstruction loss plus an $L_1$ sparsity penalty, learning a dictionary of features such that any activation is approximately a sparse linear combination of dictionary vectors.

The core ideas build on decades of work in sparse coding, neuroscience, and dictionary learning. What makes SAEs novel as an interpretability tool is the combination of scale (billion-token training runs on frontier language models), the empirical finding that the learned features are often monosemantic and human-interpretable, and the connection to the superposition hypothesis that provides a theoretical framework for why language models should have sparse, approximately-orthogonal internal feature representations in the first place.

Key takeaways from this chapter:

• Superposition and polysemanticity are the motivation: language models represent more features than they have neurons by packing multiple concepts per neuron, making individual neurons hard to interpret.
• SAE architecture uses an overcomplete dictionary ($m \gg d$) with a ReLU encoder and normalized decoder columns. The encoder and decoder are untied, allowing separate optimization of feature detection and reconstruction.
• The training loss combines MSE reconstruction with $L_1$ sparsity ($\lambda \| \mathbf{z} \|_1$), with $\lambda$ controlling the sparsity-fidelity tradeoff. The $L_1$ norm is used because $L_0$ minimization is NP-hard and non-differentiable.
• Dictionary learning is the classical framework SAEs draw from. The ReLU encoder is an amortized approximation to sparse coding, enabling efficient training at scale while sacrificing some optimality.
• Dead features are a key practical challenge. Solutions include neuron resampling, auxiliary loss, and TopK activation, with TopK increasingly preferred for large-scale training.
• Evaluation combines quantitative metrics (variance explained, loss recovered) with qualitative inspection of top-activating examples. Loss recovered is more meaningful than $R^2$ because it measures preservation of task-relevant information.
• Limitations include faithfulness concerns, feature splitting, the basis non-uniqueness problem, training instability, and the unverified assumption of the superposition hypothesis.
• Large-scale SAE results have revealed millions of interpretable features in frontier language models, giving the most detailed maps of language model representations available today, and opening new directions for understanding and auditing AI systems.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about sparse autoencoders.