Activation Steering: Vectors and Representation Engineering

If you could reach into a language model's internal computation and nudge it in a specific direction, what would you change? That question drives the field of activation steering: a family of techniques that directly modify the hidden states of a transformer at inference time to shift the model's behavior in predictable, measurable ways. Unlike prompting, which works through the model's input, or fine-tuning, which works through gradient updates to the model's weights, activation steering works through the model's internal activations, the continuous-valued vectors that flow between layers and carry the model's running representation of the input.

The appeal is immediate. Prompting is indirect: you describe what you want and hope the model interprets it correctly. Fine-tuning is expensive: it requires labeled data, GPU time, and careful hyperparameter tuning. Activation steering is direct and cheap. You find the direction in activation space that corresponds to the behavior you want, add it during inference, and the model moves. The whole pipeline can be accomplished in a few minutes with a handful of example prompts.

Activation steering connects interpretability with model control. The interpretability work covered in prior chapters gave us tools for understanding what a model has learned: attention patterns in the Attention Visualization chapter revealed which tokens the model attends to; probing classifiers located linguistic features in residual stream representations; sparse autoencoders decomposed activations into interpretable features; and mechanistic interpretability's circuit-level analysis traced computation through specific attention heads and MLP neurons. Activation steering takes that understanding one step further. Instead of merely observing what a model knows, it asks whether we can use that knowledge to change what a model does.

The core insight is deceptively simple. If a model's internal representations encode concepts as directions in activation space, then adding a vector that points in the direction of a concept should increase the model's tendency to express that concept, while subtracting it should suppress the concept. This idea, sometimes called "activation addition," lets researchers and practitioners shift model behavior without retraining, fine-tuning, or modifying weights.

The practical stakes are concrete. Activation steering has been used to suppress harmful outputs, amplify specific reasoning styles, shift a model from one persona to another, and test causal hypotheses about how internal representations influence behavior. It also raises deep questions about the nature of representation in neural networks, since the technique works best when concepts are linearly encoded, which is itself an empirically testable claim for how we understand transformer models.

This chapter covers the full field of activation steering: the theoretical grounding in the linear representation hypothesis, how steering vectors are constructed and applied, the more systematic framework of representation engineering, experimental findings, practical diagnostics, and current alignment and safety applications.

Before diving into steering mechanics, it is worth understanding the theoretical premise that makes activation steering possible in the first place. The linear representation hypothesis holds that many high-level concepts are encoded as linear directions in the activation space of a transformer's residual stream.

This is not an obvious claim. A neural network with billions of parameters and nonlinear activations at every layer could in principle encode concepts in arbitrarily complex, nonlinear ways. The geometry of high-dimensional spaces is strange. A concept might be encoded as a curved manifold, a cluster without a clear center, or a combination of dimensions that only a nonlinear function can decode. The fact that linear probing works so well, a finding we examined in the Probing Classifiers chapter, is already strong evidence that representations have substantial linear structure. But activation steering makes a stronger claim: not only are concepts linearly decodable from activations, they are linearly composable. You can add vectors and the result is meaningful.

Evidence for linearity comes from multiple directions. Word embedding arithmetic, demonstrated long ago with Word2Vec's famous "king - man + woman = queen" analogy, showed that semantic relationships can be captured by vector differences. In modern transformers, similar structure appears in the residual stream at intermediate layers. Probing classifiers trained to predict linguistic features from activations generalize well even when they are simple linear models, suggesting the features are linearly accessible. Analyses of individual attention heads have found that many heads implement operations that are close to linear functions of the query and key vectors.

The geometry implied by linear representation has several attractive properties. Concepts can be combined through vector addition. The direction from the activation for "happy" to the activation for "sad" is meaningful and stable across many contexts. The representation of a concept in position $i$ at layer $\ell$ can be decomposed into contributions from attention heads and MLP layers using the residual stream decomposition we explored in the Mechanistic Interpretability chapter.

A deeper explanation for why linear representations emerge involves the concept of superposition. Transformer hidden dimensions are limited (often 768 to a few thousand dimensions), but the number of concepts a model must represent is vastly larger. The only way to store more features than you have dimensions is to use each dimension to represent multiple features at once, a situation called superposition.

Under superposition, features are stored as nearly-orthogonal directions, and the model's computations can still be approximately linear because random high-dimensional vectors are nearly orthogonal. If you have $n$ dimensions, you can store on the order of $n^2$ quasi-orthogonal features with only small interference between them. This is why linear probing and linear steering both work: the features are spread across dimensions in a way that is recoverable by linear operations, even when the features are far more numerous than the dimensions.

The sparse autoencoder work covered in the previous chapter relates directly to this: SAEs are trying to recover the underlying features from their superposed representation, moving from the dense, superposed space to a sparse space where each dimension corresponds to one feature. Activation steering, by contrast, works directly in the superposed space. It finds a direction in the dense residual stream that corresponds to a concept, and this direction may involve contributions from many underlying features simultaneously.

Understanding activation steering also requires understanding how the residual stream works. As we covered in the Transformer Blocks chapter, the residual stream is the tensor that passes between transformer layers. Each attention head and MLP layer reads from the residual stream, performs a computation, and writes back a contribution via residual addition.

where:

• $\mathbf{h}_\ell$: the residual stream at layer $\ell$ (shape: tokens $\times$ hidden dimension)
• $\text{Attn}_\ell$: the contribution from the multi-head attention at layer $\ell$
• $\text{MLP}_\ell$: the contribution from the feedforward network at layer $\ell$

The residual addition structure means that the residual stream is a running sum of all prior contributions. Because this sum is linear, the residual stream inherits the linear properties of the individual contributions. When you inject a steering vector by adding it to $\mathbf{h}_\ell$, every subsequent layer sees the modified representation and produces outputs conditioned on it. The injection persists and propagates through the rest of the computation.

Not all concepts are linearly encoded. Some abstract or compositional concepts may require more complex representations. The linear hypothesis is a useful approximation that holds well enough to make steering tractable, but it is an approximation nonetheless. We return to this limitation later in the chapter.

A steering vector is a direction in activation space that corresponds to a target concept or behavior. The goal is to find a vector $\mathbf{v}$ such that adding $\alpha \mathbf{v}$ to the model's residual stream activations shifts its behavior toward the concept while subtracting $\alpha \mathbf{v}$ shifts away from it. The coefficient $\alpha$ controls the strength of the intervention.

Several methods exist for constructing steering vectors, each with different data requirements and properties. The choice of method depends on the clarity of the concept boundary, the number of examples available, and whether the concept is binary or continuous.

The most direct method is to collect activations from the model on two sets of prompts: one representing the "positive" end of a concept and one representing the "negative" end. The steering vector is then the mean difference between these activation sets.

Concretely, suppose you want a steering vector for the concept "refusing harmful requests." You could collect:

• Positive examples: prompts where the model should (or does) refuse, such as "How do I make a weapon? I cannot help with that."
• Negative examples: prompts where the model complies without refusal, such as "How do I bake bread? First, combine flour and water..."

For each example, you extract the residual stream activation at a specified layer and token position, then compute the mean over positive examples minus the mean over negative examples. This difference vector is the steering direction.

Mathematically, let $\mathbf{h}_i^+$ be the activation at layer $\ell$ and token position $t$ for positive example $i$, and $\mathbf{h}_j^-$ for negative example $j$. The steering vector is:

where:

• $N_+$: number of positive examples
• $N_-$: number of negative examples
• $\mathbf{h}_i^+$: residual stream activation at the chosen layer and token position for positive example $i$
• $\mathbf{h}_j^-$: residual stream activation at the chosen layer and token position for negative example $j$

The resulting vector points in the direction in activation space that most strongly distinguishes the positive concept from the negative concept, averaged over the example distribution. The averaging is important: by taking the mean over many examples, you reduce noise from idiosyncratic features of individual prompts and extract the shared directional signal.

A particularly clean variant of the contrastive method uses minimally different paired prompts. For each pair $(p^+, p^-)$, the two prompts differ in exactly the relevant concept and are otherwise matched:

• "The assistant is helpful and harmless." vs. "The assistant is harmful and deceptive."
• "Think through this carefully and reason step by step." vs. "Just give me the first answer that comes to mind."
• "The user is an expert." vs. "The user is a novice."

By controlling all variation except the concept of interest, paired prompts reduce noise in the estimated direction. The steering vector from $N$ pairs is:

where the sum is over matched pairs, and $\mathbf{h}_i^+ - \mathbf{h}_i^-$ is the difference in activations between the positive and negative prompt in pair $i$.

The paired design has an additional advantage: any bias in the model's activations that is shared between the positive and negative example in a pair cancels out. If both prompts mention "the assistant," the activation contribution from "assistant" appears in both $\mathbf{h}_i^+$ and $\mathbf{h}_i^-$, so it vanishes from their difference. What remains is the signal attributable to the differing concept.

An alternative approach is to collect activations from many diverse examples of a concept and extract the principal directions of variation using PCA. Rather than specifying positive and negative classes explicitly, you gather examples that vary along the concept dimension and let PCA find the axis of maximal variance.

This can be more reliable when concepts do not have clean binary contrast. For example, "degree of formality" is a spectrum, not a binary, and directly contrasting "very formal" with "very informal" may miss the richest variation in the middle of the distribution. PCA captures the full variance structure. However, it requires more careful data curation: the principal component corresponds to the axis of maximum variance in the collected examples, which might not align with the target concept if the data contains other strong sources of variation. The direction of the extracted component may also need to be flipped manually to align with the desired direction.

A steering vector is layer-specific and sometimes token-position-specific. Choosing the right layer matters considerably.

Early layers tend to encode low-level syntactic features and are less useful for high-level concept steering. At these layers, the residual stream still closely resembles the token embeddings, which carry mostly surface-level information. Intervening at early layers changes what information flows into the first few attention heads but may not reach the higher-level representations where the concept lives.

Middle layers in large models often encode rich semantic representations. This is where many interpretability studies have found evidence for syntactic structure, entity tracking, factual recall, and sentiment. Steering at middle layers tends to have the strongest behavioral effect on high-level concepts because the intervention happens when the representation is most concept-rich and still has many downstream layers to act on it.

Very late layers are close to the output distribution. By this point, the model is largely computing the output logits and the most abstract conceptual work has already been done. Steering here can still shift the output distribution, but the effect may be narrower and less interpretable.

Empirically, middle layers (roughly one-third to two-thirds through the network) produce the most consistent results across different models and concepts. The optimal layer can be identified by measuring the behavioral effect of steering at each candidate layer using a held-out evaluation set, or by examining the steering vector norm across layers to find where the concept is most differentiated.

The token position also matters because the residual stream carries different information at different positions. For causal language models, the last token of the prompt carries the most context because it has attended to all prior tokens. Extracting activations at the last token position and injecting at the last token position of each generated token is the most common approach. Some researchers extract from the position of a semantically relevant token when the steering concept is tightly linked to a specific word in the prompt.

Once you have a steering vector $\mathbf{v}$, applying it is straightforward. During a forward pass, after computing the residual stream activations at layer $\ell$, you add a scaled version of the vector before passing activations to the next layer:

where:

• $\mathbf{h}_\ell$: the residual stream activations at layer $\ell$ (shape: tokens $\times$ hidden dimension)
• $\alpha$: the steering coefficient, a scalar multiplier controlling intervention strength
• $\mathbf{v}$: the steering vector, a vector in the hidden dimension

Adding $\alpha \mathbf{v}$ with $\alpha > 0$ pushes activations toward the positive concept. Subtracting (using $\alpha < 0$) pushes away from the concept.

The choice of $\alpha$ is empirical and context-dependent. Small values produce subtle behavioral shifts; large values produce strong shifts but can degrade output quality or cause incoherence. A typical range for well-calibrated steering vectors is $\alpha \in [5, 30]$ when the vector has unit norm, but the right value depends on the model size, the concept, and the desired strength of intervention. This is one reason to normalize the steering vector to unit length before applying it: it makes the coefficient interpretable as a distance in activation space rather than an arbitrary scaling that depends on how the vector was constructed.

The injection can be applied at different token positions with different effects. Applying it at every position during generation creates a persistent context: every token the model generates is conditioned on the steered representation throughout. Applying it only at prompt tokens sets an initial direction that influences how the model interprets the prompt. Applying it only at generated tokens modifies generation without changing prompt processing. The most consistent approach, and the one that produces the strongest effects, is constant injection at all positions during generation.

In practice, the injection is implemented via a forward hook: a callback function registered on a specific layer that intercepts the layer's output, adds the scaled steering vector, and returns the modified output. This requires no modification to the model weights and can be added and removed dynamically. The PyTorch hook system makes this straightforward, as we will see in the implementation section.

A natural extension is to inject the steering vector at multiple layers simultaneously. If the concept is encoded across several layers, injecting at all of them may produce a stronger, more consistent effect than injection at a single layer.

However, multi-layer injection can also amplify side effects and cause interactions between the injections at different layers. If the injection at layer $\ell$ shifts activations in a way that changes what layer $\ell+1$ computes, and then another injection at $\ell+1$ adds on top of that, the combined effect may not be a simple superposition of the individual effects. Empirically, single-layer injection at a carefully chosen middle layer often works as well as multi-layer injection with less risk of degradation.

The term "activation addition" was introduced in a 2023 paper by Turner, Thiergart, Udell, Mini, and MacDiarmid, who demonstrated the technique on GPT-2-XL. Their experiments provide a clean illustration of what steering can and cannot do.

The setup was simple: construct a steering vector by subtracting activations for a "negative" prompt from activations for a "positive" prompt, then add a scaled version of this vector to all forward pass positions at a specific layer. No training, no weight updates, no optimizer. Just arithmetic on activations.

One of their most striking demonstrations involved constructing a "banana" vector by taking the difference between "I talk about bananas all the time." and "I talk about oranges all the time." When this vector was injected with a high coefficient during generation, the model began inserting banana-related content into otherwise unrelated outputs. The model would discuss history but somehow link it to bananas, or write code but reference bananas unprompted. The change extended beyond the word "banana"; the model's conceptual associations shifted, with its reasoning becoming banana-adjacent in context-appropriate ways.

The banana demonstration is deliberately whimsical to make the core point vivid: the vector really does encode something conceptual, not just lexical. If it were only encoding the word "banana," you would see the word inserted at random positions. Instead, the model integrates the banana concept into its outputs in grammatically appropriate ways, treating it as a persistent conceptual context.

More practically, Turner et al. constructed vectors for concepts like "wedding," "anger," "fearful," and "optimistic" and showed that injecting these vectors reliably shifted the model's outputs toward the relevant concept while maintaining grammatical coherence and largely preserving responses to the specific question asked.

The experiments also revealed that the technique has failure modes. At high steering coefficients, outputs can become incoherent or the model can begin to "rant" repetitively. There is a sweet spot between no effect and degraded output, and finding it requires empirical calibration per concept and per model. The sweet spot also shifts with model size: larger models tend to be more robust to high steering coefficients before their outputs degrade, suggesting that bigger models have more capacity to absorb the perturbation without losing coherence.

Activation addition and prompt engineering are both ways to influence model behavior without changing weights, but they operate through completely different mechanisms. A prompt like "Be confident and assertive in your response" works by adding text to the context window that the model attends to when generating. It relies on the model having learned to follow such instructions from its training distribution.

Activation steering bypasses the attention mechanism entirely. It modifies the representation directly, without the model "reading" a steering instruction. This has several implications. Steering is more robust to prompts that would override or dilute the steering instruction: adding a steering vector cannot be "ignored" by the model the way a prompt instruction can. Steering also does not consume context window tokens, making it free in terms of the prompt budget. On the other hand, steering does not benefit from the model's capacity to reason about and interpret complex instructions; it is a blunt geometric intervention.

While activation addition focuses on finding and applying individual steering vectors, representation engineering (RepE), introduced by Zou, Phan, Chen, Campbell, Guo, Bhatt, Nowak, Liu, Carlin, and Steinhardt in 2023, provides a more systematic framework for understanding and manipulating the internal representations of language models.

The central idea of representation engineering is to treat control as a measurement problem. Rather than directly constructing vectors from contrast pairs and applying them, RepE first establishes a reading direction for a concept: a direction in the model's representation space that can be used to measure the model's internal state with respect to that concept, and then separately establishes a control direction that can be used to shift that state.

The separation between reading and writing is a key conceptual contribution. In activation addition, you construct a vector and apply it, checking only the behavioral output to see if it worked. Representation engineering insists on verifying that the reading direction captures the intended concept before using it for control. This makes the approach more rigorous and more interpretable.

A reading direction is extracted by:

1. Constructing a dataset of contrast pairs that vary along the concept dimension, similar to the contrastive activation difference method.
2. Running the model on these pairs and collecting activations from a target layer.
3. Applying PCA to the collected activation differences to find the dominant direction of variation.
4. Normalizing this direction to unit length to get the reading vector $\hat{\mathbf{r}}$.

The reading vector $\hat{\mathbf{r}}$ can then be used to score any activation: a high inner product $\mathbf{h} \cdot \hat{\mathbf{r}}$ indicates the model is strongly in the "positive" region for the concept. This score is sometimes called the "representation activity" of the concept at a given layer and token position.

This reading operation is useful on its own as a form of interpretability. You can visualize how a model's internal representation of a concept like "honesty" or "fear" evolves across layers during a specific prompt, or compare how different model variants represent the same concept. If you feed the model a prompt that gradually escalates toward a policy violation, the representation activity of "harmlessness" should decrease systematically as the escalation progresses, and you can observe exactly where in the layer stack that decrease occurs.

The control direction is the same (or a closely related) vector used to modify activations. Because representation engineering explicitly separates reading from writing, it becomes possible to:

• Measure the model's internal state before an intervention
• Apply a calibrated intervention
• Measure the state again to verify the intervention had the intended effect

This feedback loop is important for reliable control. A concept vector that successfully reads the internal state might not produce the intended behavioral shift if applied naively. There may be a gap between the direction in representation space that best discriminates the concept and the direction that most effectively shifts behavior. Representation engineering's framework encourages verification of both the reading accuracy and the control effect, rather than assuming they are the same.

The original representation engineering paper demonstrated control over a range of high-stakes model behaviors.

Honesty steering reduced the rate of sycophantic responses in TruthfulQA-style evaluations. When the model was asked a question and given incorrect context that it was trained to agree with, honesty steering reduced the rate of agreement with the false context, pushing the model toward more accurate responses.

Emotion steering produced outputs that human raters consistently identified as reflecting the target emotion. Steering toward "happy," "sad," "fearful," or "surprised" representations shifted the content of the model's responses as well as their linguistic tone and related associations.

Harmlessness steering reduced the rate of harmful output generation. When the model was prompted with adversarial inputs designed to elicit policy violations, steering toward the "harmless" representation made the attacks less successful, without fine-tuning or modifying the model.

Memorization steering increased the rate at which the model reproduced verbatim text from its training data. By steering toward representations associated with memorized content, the researchers could increase the probability that the model would output a specific memorized sequence, giving evidence that memorized content is represented in a specific region of activation space.

Critically, all of these interventions were applied only at inference time, with no weight updates. The model's behavior was shifted purely by modifying the stream of activations during generation.

One of the most valuable analyses in the RepE framework is probing the reading direction across all layers to understand at which point in the computation a concept "emerges." For a concept like "honesty," you can compute the representation activity $\mathbf{h}_\ell \cdot \hat{\mathbf{r}}$ at each layer $\ell$ and plot how it evolves across the forward pass.

These layer-by-layer profiles reveal interpretable structure. Some concepts are first encoded in early layers and remain stable. Others are absent in early layers and appear sharply at a specific transition, suggesting they are computed rather than directly inherited from the input. Some concepts fluctuate across layers, suggesting multiple computational passes over the concept as the model refines its representation.

This analysis also informs layer selection for control: the optimal control layer is typically the first layer at which the reading direction becomes strongly predictive, not the last. Intervening early gives subsequent layers the maximum opportunity to incorporate the steered representation.

Let us implement activation steering from scratch using a small transformer model via Hugging Face's Transformers library. We will build a steering vector for a simple concept, diagnose its geometry, and demonstrate applying it during generation.

We begin with library imports and model setup.

GPT-2 has 12 transformer layers and a hidden dimension of 768. We will target layer 8, approximately two-thirds through the network, for our steering vector. Middle-to-late layers tend to hold the richest semantic representations because they have processed enough context to form abstract concepts but still have downstream layers that can act on the modified representation.

We write a helper function to extract residual stream activations at a specified layer and token position.

The function runs the model in inference mode with no gradient computation and retrieves the hidden states tensor from the output. Because the output_hidden_states=True flag was set during model loading, the model returns a tuple of activation tensors: one for the embedding layer and one for each transformer block. We index layer_idx + 1 to skip past the embedding layer output.

Notice that we extract from token_position=-1, the last token. In a causal language model, the last token has attended to all prior tokens through causal self-attention, so its residual stream carries a compressed representation of the entire context up to that point. This is the richest single position from which to extract a concept-level signal.

With activation extraction in place, we construct a steering vector using contrasting prompts.

The steering vector has shape (768,), one value per hidden dimension. The raw (unnormalized) norm reflects the distance in activation space between the average "confident" activation and the average "uncertain" activation. Normalization ensures that the coefficient $\alpha$ controls the intervention strength in a consistent unit, regardless of the raw vector magnitude, so that the same value of $\alpha$ means the same geometric displacement for every concept vector.

To apply the steering vector during a forward pass, we hook into the model and modify activations at the target layer. We compare next-token probabilities with and without the intervention to verify the steering effect quickly, without the computational cost of full text generation.

The table shows how the steering vector shifts the probability distribution over next tokens. Confident steering (positive alpha) raises the probability of assertive continuations while uncertain steering (negative alpha) raises the probability of hedged ones, confirming that the vector captures a real semantic direction in the model's activation space. This rapid token-probability check is a standard first-pass diagnostic before investing in full generation experiments.

A useful diagnostic is to check how similar different steering vectors are to each other. If a vector constructed for "confident" is nearly identical to one for "positive tone," the two concepts may be entangled in the model's representation space, meaning steering one will also steer the other.

Cosine similarity near zero means the two vectors are pointing in largely unrelated directions, suggesting they capture distinct aspects of the model's representation space. Values close to 1.0 indicate that steering for one concept will also strongly steer for the other. This diagnostic is essential before deploying a steering vector: if your "harmlessness" vector is highly similar to your "passivity" vector, you may be inadvertently making the model passive rather than harmless.

The key parameters for activation steering are:

• target_layer: The layer where the steering vector is extracted and applied. Middle-to-late layers (roughly 60-75% through the network) typically work best for semantic concepts.
• alpha: The scaling coefficient controlling intervention strength. Too small and the effect is negligible; too large and outputs become incoherent.
• token_position: The token position from which activations are extracted when building the vector. The last token (-1) is the standard choice for causal language models.
• n_prompts: The number of positive and negative prompts used to estimate the steering direction. More examples reduce noise in the direction estimate.

To build intuition for what steering does geometrically, we can visualize how activations change as we vary the steering coefficient. We project the high-dimensional activations onto the steering direction and examine how the alignment changes with $\alpha$.

The heatmap reveals the structure of the model's semantic geometry at layer 8. Pairs with low off-diagonal similarity (near 0) are good candidates for independent simultaneous steering: you could apply both a "confident" vector and a "formal" vector at the same time with minimal interaction. Pairs with high similarity are entangled and require care: the intervention will affect both concepts whether you intend it to or not.

The ability to directly modify model behavior at inference time matters for AI safety and alignment. Activation steering allows interventions that are otherwise difficult or expensive to achieve through fine-tuning, and it opens up new possibilities for monitoring and controlling deployed models.

One of the most direct applications is suppressing harmful behaviors. Constructing a steering vector that points from "harmful/compliant" activations to "harmless/refusing" activations and applying it during generation can reduce the rate of policy-violating outputs.

The appeal of this approach over fine-tuning is its speed and flexibility. A steering vector can be constructed from a small set of examples in minutes, applied selectively to certain prompt types, and adjusted in real time without modifying model weights. If a new category of harmful content is identified, you can construct a new steering vector immediately rather than waiting for a fine-tuning run to complete.

The risk is that naively constructed steering vectors can suppress legitimate outputs along with harmful ones. A steering vector for "refusals" might also suppress legitimate uncertainty expression or hedging, making the model appear inappropriately confident when it has been steered toward refusal language. Evaluating steering vectors on held-out benchmarks for both the target behavior and potential side effects is essential before deployment.

Representation engineering can be used to monitor the model's internal state during inference. Research has shown that when a jailbreak is in progress, the model's internal representations shift in detectable ways before the policy-violating output is generated. The model's "harmlessness" representation activity decreases measurably as a jailbreak prompt escalates through a multi-turn conversation.

This creates the possibility of real-time monitoring: detect when activations drift toward a "jailbroken" region and intervene before the output is produced, either by rejecting the response or by applying a corrective steering vector to pull the model back toward the safe region. This approach is proactive rather than reactive: it can catch violations before they occur rather than filtering outputs after generation.

Several researchers have demonstrated that steering vectors can reduce sycophancy, the tendency of language models to agree with users rather than provide accurate information. A steering vector pointing from "agreeable/sycophantic" representations to "honest/accurate" representations can shift the model toward providing more truthful assessments even when the user has expressed a contrary view.

This application is particularly interesting because it targets an internal representation rather than a surface behavior. Training on examples of "don't agree with wrong statements" might cause the model to learn to avoid certain surface patterns in its outputs without changing its internal confidence calibration. Activation steering targets the internal state directly, potentially producing a more robust intervention.

The limitation is that "honesty" as a concept may not have a single stable direction in representation space across all contexts. A model might represent "honesty about factual claims" in a different region than "honesty about emotional states" or "honesty about the model's own uncertainty." Building a steering vector that works across these different contexts requires diverse, carefully balanced contrast pairs.

Steering vectors can encode high-level behavioral dispositions. A "helpful assistant" vector, a "cautious expert" vector, or a "creative collaborator" vector can be constructed and applied to shift the model into a particular behavioral mode. This is lighter-weight than fine-tuning for persona adaptation and more interpretable: the steering direction can be inspected to understand what representation the model is being pushed toward.

For organizations deploying language models in specific contexts, this means that a single base model can be adapted to different personas at inference time by switching which steering vectors are active, without maintaining separate fine-tuned model checkpoints for each persona.

Beyond practical safety applications, activation steering is a powerful research tool for testing causal hypotheses about how language models work. Correlation-based interpretability methods (probing, attention visualization) can identify that a certain representation is associated with a behavior, but they cannot determine whether the representation causes the behavior.

Steering vectors provide a causal test. If you hypothesize that the direction found by probing for "truthfulness" is causally responsible for honest outputs, you can test this by applying a steering vector along that direction and checking whether honest output increases. If steering in the predicted direction produces the predicted behavioral change, that is evidence for a causal relationship. If not, the representation may be a read-out of behavior rather than its cause.

This causal verification is what makes representation engineering valuable as both a scientific and practical tool. It brings the experimental logic of causal interventions to the study of internal model representations.

A natural question is whether steering vectors compose linearly. If you add both a "confident" vector and a "formal" vector to the residual stream simultaneously, do you get output that is both confident and formal, as if the two concepts superpose independently?

Empirical evidence suggests that compositional steering works reasonably well when the two vectors have low cosine similarity (are nearly orthogonal), and breaks down when the vectors are entangled. For orthogonal concepts, the behavioral effects are approximately additive: the model exhibits both steered properties simultaneously. For entangled concepts, the combined intervention can produce unexpected effects as the two steers interact in the model's nonlinear computation.

This compositional behavior has a practical implication: the pairwise similarity heatmap is a diagnostic and a planning tool. Before combining steering vectors, check their similarity. Low similarity pairs can be combined safely. High similarity pairs should be used exclusively or with careful calibration.

Soft prompting, covered in the Prompt Tuning chapter, is another inference-time technique for shifting model behavior without modifying weights. In soft prompting, a set of learned continuous token embeddings is prepended to the prompt, shifting the model's behavior through the input. How does this compare to activation steering?

The key difference is where in the model the intervention occurs. Soft prompts operate at the input layer and influence the model entirely through the attention mechanism: all layers attend to the soft prompt tokens and are conditioned on them. Activation steering operates directly on the residual stream at a specific layer, bypassing the attention mechanism for layers upstream of the injection point.

This difference has practical consequences. Soft prompts require gradient-based optimization to learn: you need labeled data and a training procedure. Steering vectors can be constructed analytically from a few examples with no gradient computation. Soft prompts are learned jointly with the task distribution and can be highly effective at task adaptation. Steering vectors are more interpretable (you know what concept direction you are steering in) and more flexible (no training required).

Neither approach is universally superior. For task adaptation with labeled data available, soft prompts may outperform steering vectors. For rapid behavioral adjustment and interpretable control without training data, steering vectors are often preferred.

Activation steering and fine-tuning both aim to shift model behavior, but they operate through fundamentally different mechanisms. Fine-tuning changes the model's weights through gradient descent on a training objective, producing a persistent change that survives after the training run. The behavioral change is embedded in the parameters.

Activation steering changes the model's activations during inference, producing a behavioral shift that exists only when the steering vector is applied. Remove the vector and the model returns to its base behavior. This is both a limitation and an advantage. The limitation is that steering cannot encode new factual knowledge or teach new skills, since the weights are unchanged. The advantage is that steering is immediately reversible and can be applied conditionally, for example, applying a "cautious" steering vector only for high-stakes queries.

Fine-tuning can also be understood through the lens of activation steering. What does fine-tuning do to the model's activations? Research has shown that fine-tuning on a behavioral objective tends to shift the model's representations in the direction that activation steering would move them, suggesting that steering provides a rough, gradient-free approximation to fine-tuning in the space of activations. This connection is not yet fully understood and is an active area of research.

Activation steering is a powerful technique, but it comes with significant limitations that any practitioner should understand.

The entire framework rests on the assumption that concepts are linearly encoded in activation space. For many concepts this works well, but for compositional or contextual concepts the assumption breaks down. "Being sarcastic about a compliment" cannot be represented as the sum of "sarcasm" and "compliment" vectors. As concepts become more context-dependent and combinatorially complex, the linear representation hypothesis becomes less reliable.

Steering effectiveness is highly sensitive to the choice of target layer and scaling coefficient. The same vector applied at different layers can produce qualitatively different effects, or no effect at all. There is no principled rule for choosing the right layer; it requires empirical search, and the right answer depends on the model architecture, the concept, and the specific prompt distribution. This sensitivity means that steering vectors developed on one model do not transfer reliably to architecturally different models, even if those models have similar capabilities.

Adding a vector to the residual stream affects the target concept and other aspects of computation that share representation space with that concept. A vector pushing toward "confident" outputs might also shift the model away from representations associated with qualified uncertainty expression, or toward representations associated with other concepts that happen to be geometrically close to "confident" in this model. Large steering coefficients amplify these side effects and can cause the model to enter a repetitive or incoherent generation mode where the steered direction overwhelms all other computation.

The quality of a steering vector is only as good as the contrast pairs used to construct it. If the positive and negative examples differ along dimensions other than the target concept, the vector will capture those differences as well. A "refusal" vector built from examples where the positive examples are all formal and the negative examples are all casual will encode formality along with refusal. Careful data collection and curation, using many diverse pairs that vary along the target concept while holding other dimensions roughly constant, reduces this confounding, but it requires expert judgment and is not easily automated.

A steering vector that reliably shifts behavior on a validation set of prompts may not generalize to qualitatively different prompts. If the construction dataset is narrow, the vector might only work well for prompts that are distributionally similar to the examples it was built from. This is particularly concerning for safety applications where adversarial prompts are specifically designed to be out-of-distribution.

Finally, steering changes the model's outputs but not its weights. Once the steering vector is removed, the model returns to its base behavior. This is useful for inference-time control but is not a substitute for training-based alignment when persistent behavior change is the goal. Activation steering is best understood as a complement to fine-tuning and RLHF, not a replacement.

Activation steering provides a principled way to modify language model behavior at inference time by injecting directional vectors into the model's residual stream. The technique rests on the linear representation hypothesis: many concepts are encoded as directions in activation space, making them addable and subtractable through simple vector arithmetic.

Steering vectors are constructed by computing the mean activation difference between examples of a concept and examples of its contrast. The contrastive activation difference method averages activations over positive and negative prompt sets. Paired prompts reduce noise by matching example pairs that differ only in the target concept. PCA-based methods extract the dominant direction of variation from a more diverse set of examples.

Once constructed, a steering vector is applied by registering a forward hook that adds a scaled version of the vector to the residual stream at a target layer. The scaling coefficient $\alpha$ controls intervention strength: positive values push toward the concept, negative values push away. Normalizing the vector to unit length makes $\alpha$ interpretable as a distance in activation space.

The representation engineering framework extends activation addition into a systematic approach with explicit representation reading (measuring internal states with a reading vector) and representation writing (modifying them with a control vector), enabling more controlled and verifiable interventions. The separation between reading and writing allows verification that the vector is capturing the intended concept before it is used for control.

Key diagnostics include the layer-by-layer steering vector norm (to identify the optimal target layer), the cosine similarity between steering directions (to check for entanglement between concepts), and the next-token probability comparison before and after steering (to rapidly verify that the vector captures a real semantic direction).

Applications span reducing harmful outputs, improving honesty and reducing sycophancy, enabling real-time jailbreak monitoring, and testing causal hypotheses about how internal representations drive behavior. Limitations include dependence on the linearity of representation, sensitivity to layer and coefficient choice, side effects on related concepts, and lack of persistence once steering is removed.

As we move into the next part of this book, the ability to both read and write model representations connects activation steering to the broader challenge of reliably aligning language model behavior with human intentions. Activation steering is not the final answer to alignment, but it is one of the most direct and interpretable tools we currently have for making the connection between internal representation and external behavior explicit and manipulable.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about activation steering.