Feature Interpretation › Feature Geometry

Beyond understanding individual features in isolation, a powerful complementary approach is to study the geometric structure of the feature space as a whole. Features are vectors in $\mathbb{R}^d$, and their relative positions, angles, and clusters carry information about the model's knowledge organization.

Two features $\mathbf{f}_i$ and $\mathbf{f}_j$ (taken as the decoder weight vectors) that represent semantically similar concepts tend to have higher cosine similarity:

$$\cos(\mathbf{f}_i, \mathbf{f}_j) = \frac{\mathbf{f}_i \cdot \mathbf{f}_j}{\|\mathbf{f}_i\| \|\mathbf{f}_j\|}$$

For example, features representing different European capital cities will be closer to each other in feature space than to features representing programming language constructs. This geometric proximity is not accidental: it reflects the model's learned associative structure, shaped by the co-occurrence statistics in the training data.

This observation motivates clustering the feature space to discover "feature families": groups of semantically related features that occupy nearby regions of activation space. Clustering can be done with any standard algorithm (k-means, hierarchical clustering, UMAP-based visualization). The resulting clusters often correspond to recognizable semantic domains: a cluster of medical features, a cluster of mathematical notation features, a cluster of proper-name features for a particular language or culture.

Feature families also have a practical use in circuit analysis. If you are trying to understand a model's behavior on a specific topic (say, legal reasoning), you can start by finding the cluster of features in the "legal domain" family and then trace how those features interact with each other and with features in adjacent families.

Some of the most striking results in feature geometry concern linear subspaces. Just as in classic word embedding research we could find vector analogies like "king - man + woman = queen," SAE features sometimes exhibit analogous relational structure, but at a higher level of abstraction.

For instance, if you find a feature for "Paris is a capital city" and a feature for "France is the country containing Paris," there may be a consistent vector offset between features representing "X is a capital city" and "X is a major non-capital city" for various values of X. These structured relationships in feature space reflect the model's learned knowledge of systematic world-knowledge patterns: the same offset that relates "Paris" features to "Lyon" features also relates "Berlin" features to "Hamburg" features.

The existence of such relational structure has implications for how we think about what models have learned. It suggests that the model has organized facts into relational schemas represented as geometric structure in activation space, rather than memorizing individual facts in isolation. Discovering and characterizing these relational structures is one of the more intellectually exciting frontiers of feature geometry research.

As you train SAEs with different dictionary sizes on the same model, you observe a phenomenon called feature splitting and feature absorption. When you double the dictionary size, previously identified features often split into two or more finer-grained features that together cover the activation variance of the original feature. Conversely, when you reduce the dictionary size, related features merge into a single coarser feature.

This behavior provides a natural multi-resolution view of the model's representations. At low dictionary resolution, you see coarse semantic categories. At high resolution, you see fine-grained distinctions within those categories. The optimal dictionary size for a given interpretability question depends on how fine-grained your analysis needs to be: for understanding broad model behavior, a smaller dictionary may suffice; for understanding subtle within-category distinctions (for example, different registers of formal English), a larger dictionary is necessary.