Activation Steering: Steering Vectors and Representation Engineering › Representation Engineering

While activation addition focuses on finding and applying individual steering vectors, representation engineering (RepE), introduced by Zou, Phan, Chen, Campbell, Guo, Bhatt, Nowak, Liu, Carlin, and Steinhardt in 2023, provides a more systematic framework for understanding and manipulating the internal representations of language models.

The central idea of representation engineering is to treat control as a measurement problem. Rather than directly constructing vectors from contrast pairs and applying them, RepE first establishes a reading direction for a concept: a direction in the model's representation space that can be used to measure the model's internal state with respect to that concept, and then separately establishes a control direction that can be used to shift that state.

The separation between reading and writing is a key conceptual contribution. In activation addition, you construct a vector and apply it, checking only the behavioral output to see if it worked. Representation engineering insists on verifying that the reading direction captures the intended concept before using it for control. This makes the approach more rigorous and more interpretable.

A reading direction is extracted by:

1. Constructing a dataset of contrast pairs that vary along the concept dimension, similar to the contrastive activation difference method.
2. Running the model on these pairs and collecting activations from a target layer.
3. Applying PCA to the collected activation differences to find the dominant direction of variation.
4. Normalizing this direction to unit length to get the reading vector $\hat{\mathbf{r}}$.

The reading vector $\hat{\mathbf{r}}$ can then be used to score any activation: a high inner product $\mathbf{h} \cdot \hat{\mathbf{r}}$ indicates the model is strongly in the "positive" region for the concept. This score is sometimes called the "representation activity" of the concept at a given layer and token position.

This reading operation is useful on its own as a form of interpretability. You can visualize how a model's internal representation of a concept like "honesty" or "fear" evolves across layers during a specific prompt, or compare how different model variants represent the same concept. If you feed the model a prompt that gradually escalates toward a policy violation, the representation activity of "harmlessness" should decrease systematically as the escalation progresses, and you can observe exactly where in the layer stack that decrease occurs.

The control direction is the same (or a closely related) vector used to modify activations. Because representation engineering explicitly separates reading from writing, it becomes possible to:

• Measure the model's internal state before an intervention
• Apply a calibrated intervention
• Measure the state again to verify the intervention had the intended effect

This feedback loop is important for reliable control. A concept vector that successfully reads the internal state might not produce the intended behavioral shift if applied naively. There may be a gap between the direction in representation space that best discriminates the concept and the direction that most effectively shifts behavior. Representation engineering's framework encourages verification of both the reading accuracy and the control effect, rather than assuming they are the same.

The original representation engineering paper demonstrated control over a range of high-stakes model behaviors.

Honesty steering reduced the rate of sycophantic responses in TruthfulQA-style evaluations. When the model was asked a question and given incorrect context that it was trained to agree with, honesty steering reduced the rate of agreement with the false context, pushing the model toward more accurate responses.

Emotion steering produced outputs that human raters consistently identified as reflecting the target emotion. Steering toward "happy," "sad," "fearful," or "surprised" representations shifted the content of the model's responses as well as their linguistic tone and related associations.

Harmlessness steering reduced the rate of harmful output generation. When the model was prompted with adversarial inputs designed to elicit policy violations, steering toward the "harmless" representation made the attacks less successful, without fine-tuning or modifying the model.

Memorization steering increased the rate at which the model reproduced verbatim text from its training data. By steering toward representations associated with memorized content, the researchers could increase the probability that the model would output a specific memorized sequence, giving evidence that memorized content is represented in a specific region of activation space.

Critically, all of these interventions were applied only at inference time, with no weight updates. The model's behavior was shifted purely by modifying the stream of activations during generation.

One of the most valuable analyses in the RepE framework is probing the reading direction across all layers to understand at which point in the computation a concept "emerges." For a concept like "honesty," you can compute the representation activity $\mathbf{h}_\ell \cdot \hat{\mathbf{r}}$ at each layer $\ell$ and plot how it evolves across the forward pass.

These layer-by-layer profiles reveal interpretable structure. Some concepts are first encoded in early layers and remain stable. Others are absent in early layers and appear sharply at a specific transition, suggesting they are computed rather than directly inherited from the input. Some concepts fluctuate across layers, suggesting multiple computational passes over the concept as the model refines its representation.

This analysis also informs layer selection for control: the optimal control layer is typically the first layer at which the reading direction becomes strongly predictive, not the last. Intervening early gives subsequent layers the maximum opportunity to incorporate the steered representation.