Activation Steering: Steering Vectors and Representation Engineering › Advanced Topics

A natural question is whether steering vectors compose linearly. If you add both a "confident" vector and a "formal" vector to the residual stream simultaneously, do you get output that is both confident and formal, as if the two concepts superpose independently?

Empirical evidence suggests that compositional steering works reasonably well when the two vectors have low cosine similarity (are nearly orthogonal), and breaks down when the vectors are entangled. For orthogonal concepts, the behavioral effects are approximately additive: the model exhibits both steered properties simultaneously. For entangled concepts, the combined intervention can produce unexpected effects as the two steers interact in the model's nonlinear computation.

This compositional behavior has a practical implication: the pairwise similarity heatmap is a diagnostic and a planning tool. Before combining steering vectors, check their similarity. Low similarity pairs can be combined safely. High similarity pairs should be used exclusively or with careful calibration.

Soft prompting, covered in the Prompt Tuning chapter, is another inference-time technique for shifting model behavior without modifying weights. In soft prompting, a set of learned continuous token embeddings is prepended to the prompt, shifting the model's behavior through the input. How does this compare to activation steering?

The key difference is where in the model the intervention occurs. Soft prompts operate at the input layer and influence the model entirely through the attention mechanism: all layers attend to the soft prompt tokens and are conditioned on them. Activation steering operates directly on the residual stream at a specific layer, bypassing the attention mechanism for layers upstream of the injection point.

This difference has practical consequences. Soft prompts require gradient-based optimization to learn: you need labeled data and a training procedure. Steering vectors can be constructed analytically from a few examples with no gradient computation. Soft prompts are learned jointly with the task distribution and can be highly effective at task adaptation. Steering vectors are more interpretable (you know what concept direction you are steering in) and more flexible (no training required).

Neither approach is universally superior. For task adaptation with labeled data available, soft prompts may outperform steering vectors. For rapid behavioral adjustment and interpretable control without training data, steering vectors are often preferred.

Activation steering and fine-tuning both aim to shift model behavior, but they operate through fundamentally different mechanisms. Fine-tuning changes the model's weights through gradient descent on a training objective, producing a persistent change that survives after the training run. The behavioral change is embedded in the parameters.

Activation steering changes the model's activations during inference, producing a behavioral shift that exists only when the steering vector is applied. Remove the vector and the model returns to its base behavior. This is both a limitation and an advantage. The limitation is that steering cannot encode new factual knowledge or teach new skills, since the weights are unchanged. The advantage is that steering is immediately reversible and can be applied conditionally, for example, applying a "cautious" steering vector only for high-stakes queries.

Fine-tuning can also be understood through the lens of activation steering. What does fine-tuning do to the model's activations? Research has shown that fine-tuning on a behavioral objective tends to shift the model's representations in the direction that activation steering would move them, suggesting that steering provides a rough, gradient-free approximation to fine-tuning in the space of activations. This connection is not yet fully understood and is an active area of research.