Activation Steering: Steering Vectors and Representation Engineering › The Linear Representation Hypothesis

Before diving into steering mechanics, it is worth understanding the theoretical premise that makes activation steering possible in the first place. The linear representation hypothesis holds that many high-level concepts are encoded as linear directions in the activation space of a transformer's residual stream.

This is not an obvious claim. A neural network with billions of parameters and nonlinear activations at every layer could in principle encode concepts in arbitrarily complex, nonlinear ways. The geometry of high-dimensional spaces is strange. A concept might be encoded as a curved manifold, a cluster without a clear center, or a combination of dimensions that only a nonlinear function can decode. The fact that linear probing works so well, a finding we examined in the Probing Classifiers chapter, is already strong evidence that representations have substantial linear structure. But activation steering makes a stronger claim: not only are concepts linearly decodable from activations, they are linearly composable. You can add vectors and the result is meaningful.

Evidence for linearity comes from multiple directions. Word embedding arithmetic, demonstrated long ago with Word2Vec's famous "king - man + woman = queen" analogy, showed that semantic relationships can be captured by vector differences. In modern transformers, similar structure appears in the residual stream at intermediate layers. Probing classifiers trained to predict linguistic features from activations generalize well even when they are simple linear models, suggesting the features are linearly accessible. Analyses of individual attention heads have found that many heads implement operations that are close to linear functions of the query and key vectors.

The geometry implied by linear representation has several attractive properties. Concepts can be combined through vector addition. The direction from the activation for "happy" to the activation for "sad" is meaningful and stable across many contexts. The representation of a concept in position $i$ at layer $\ell$ can be decomposed into contributions from attention heads and MLP layers using the residual stream decomposition we explored in the Mechanistic Interpretability chapter.

A deeper explanation for why linear representations emerge involves the concept of superposition. Transformer hidden dimensions are limited (often 768 to a few thousand dimensions), but the number of concepts a model must represent is vastly larger. The only way to store more features than you have dimensions is to use each dimension to represent multiple features at once, a situation called superposition.

Under superposition, features are stored as nearly-orthogonal directions, and the model's computations can still be approximately linear because random high-dimensional vectors are nearly orthogonal. If you have $n$ dimensions, you can store on the order of $n^2$ quasi-orthogonal features with only small interference between them. This is why linear probing and linear steering both work: the features are spread across dimensions in a way that is recoverable by linear operations, even when the features are far more numerous than the dimensions.

The sparse autoencoder work covered in the previous chapter relates directly to this: SAEs are trying to recover the underlying features from their superposed representation, moving from the dense, superposed space to a sparse space where each dimension corresponds to one feature. Activation steering, by contrast, works directly in the superposed space. It finds a direction in the dense residual stream that corresponds to a concept, and this direction may involve contributions from many underlying features simultaneously.

Understanding activation steering also requires understanding how the residual stream works. As we covered in the Transformer Blocks chapter, the residual stream is the tensor that passes between transformer layers. Each attention head and MLP layer reads from the residual stream, performs a computation, and writes back a contribution via residual addition.

$$\mathbf{h}_{\ell+1} = \mathbf{h}_\ell + \text{Attn}_\ell(\mathbf{h}_\ell) + \text{MLP}_\ell(\mathbf{h}_\ell)$$

where:

• $\mathbf{h}_\ell$: the residual stream at layer $\ell$ (shape: tokens $\times$ hidden dimension)
• $\text{Attn}_\ell$: the contribution from the multi-head attention at layer $\ell$
• $\text{MLP}_\ell$: the contribution from the feedforward network at layer $\ell$

The residual addition structure means that the residual stream is a running sum of all prior contributions. Because this sum is linear, the residual stream inherits the linear properties of the individual contributions. When you inject a steering vector by adding it to $\mathbf{h}_\ell$, every subsequent layer sees the modified representation and produces outputs conditioned on it. The injection persists and propagates through the rest of the computation.

Not all concepts are linearly encoded. Some abstract or compositional concepts may require more complex representations. The linear hypothesis is a useful approximation that holds well enough to make steering tractable, but it is an approximation nonetheless. We return to this limitation later in the chapter.