Sparse Autoencoders › Limitations and Practical Challenges

Sparse autoencoders are a powerful tool, but they come with several important limitations that constrain how we should interpret their outputs.

Faithfulness vs. completeness: An SAE can reconstruct activations accurately without the learned features being the actual computational primitives the model uses. The SAE optimizes reconstruction loss, not interpretability. A feature that fires for "city names" might not correspond to any discrete circuit in the model; it could be an artifact of how the SAE decomposes a continuous representation. Reconstructing an activation well is necessary but not sufficient for the features to be causally meaningful. This is perhaps the deepest limitation: the SAE gives you a decomposition, but it cannot guarantee that decomposition reflects the model's internal organization rather than an alternative basis that is statistically equivalent for reconstruction purposes.

The basis non-uniqueness problem: If the true features in a language model's representations form a nearly orthogonal dictionary, then the SAE will reliably recover them. But if the model's representations are smooth manifolds or continuous distributions rather than discrete, sparse features, then any overcomplete dictionary that spans the manifold will achieve good reconstruction, and the specific features found depend on the initialization, the $\lambda$ value, and the training order rather than on anything fundamental about the model. Different SAEs trained on the same activations may find different feature dictionaries that are equally good by reconstruction metrics but interpret the data differently.

Feature splitting and merging: As the dictionary size increases, a single concept may be split across multiple features (e.g., "fruit" splits into "tropical fruit", "citrus fruit", "stone fruit"). Conversely, small dictionaries may merge related but distinct concepts. There is no canonical granularity for features, and the right granularity likely depends on the downstream application. Consequently, comparing features across SAEs of different sizes requires care, and "feature counts" are not a stable measure of representational complexity.

Training instability: SAE training is sensitive to the $\lambda$ hyperparameter, the learning rate, and the decoder normalization schedule. Small changes can cause feature collapse, dead features, or poor reconstruction. Practitioners often train multiple SAEs with different hyperparameters and select the best based on the Pareto frontier of reconstruction quality vs. sparsity. The sensitivity is partly inherent to the $L_1$-regularized optimization field, which has many local minima, and partly due to the interaction between the dead feature problem and the decoder normalization.

Computational cost: Training SAEs on large models requires collecting billions of activation vectors. For frontier language models with thousands of layers, training SAEs for every layer and hookpoint requires enormous computational resources. The Gemma Scope project, for example, used significant Google TPU time to train SAEs across all layers of the Gemma 2 model family. This scale requirement means that analysis of SAEs across all layers and hookpoints of frontier models is accessible primarily to large research organizations, limiting independent verification of results.

The superposition hypothesis may be wrong: SAEs are motivated by the assumption that model representations are superpositions of sparse, monosemantic features. But this is a hypothesis, not an established fact. If models use distributed representations that are not sparse in any basis, SAEs would learn features that are artifacts of the decomposition rather than computational building blocks. The striking results from large-scale SAE analysis are suggestive but not conclusive, because they are difficult to verify independently. We know that SAEs produce interpretable features, but we do not know whether those features constitute a complete and non-redundant description of what the model is computing.

The labeling problem: Even when SAE features are coherent, labeling them at scale is expensive. A million-feature SAE requires a million feature labels. Automated labeling using language models can provide labels quickly but at lower quality than human annotation, and the quality is difficult to verify systematically. Errors in feature labels can propagate to downstream analyses that treat those labels as ground truth.

Despite these limitations, SAEs have produced striking results. Anthropic's analysis of Claude Sonnet's SAE features identified millions of features with coherent semantics, including features for famous individuals, programming concepts, emotional states, and many more. The feature for the name "Michael Jordan" activated across many distinct contexts, from basketball to statistics to music, suggesting the SAE had found a representation encoding the concept of a specific person rather than just a surface-level token pattern. These results demonstrate that SAEs are revealing something real about language model representations, even if the full theoretical picture of why they work so well remains incomplete.