Activation Steering: Steering Vectors and Representation Engineering › Limitations and Open Questions

Activation steering is a powerful technique, but it comes with significant limitations that any practitioner should understand.

The entire framework rests on the assumption that concepts are linearly encoded in activation space. For many concepts this works well, but for compositional or contextual concepts the assumption breaks down. "Being sarcastic about a compliment" cannot be represented as the sum of "sarcasm" and "compliment" vectors. As concepts become more context-dependent and combinatorially complex, the linear representation hypothesis becomes less reliable.

Steering effectiveness is highly sensitive to the choice of target layer and scaling coefficient. The same vector applied at different layers can produce qualitatively different effects, or no effect at all. There is no principled rule for choosing the right layer; it requires empirical search, and the right answer depends on the model architecture, the concept, and the specific prompt distribution. This sensitivity means that steering vectors developed on one model do not transfer reliably to architecturally different models, even if those models have similar capabilities.

Adding a vector to the residual stream affects the target concept and other aspects of computation that share representation space with that concept. A vector pushing toward "confident" outputs might also shift the model away from representations associated with qualified uncertainty expression, or toward representations associated with other concepts that happen to be geometrically close to "confident" in this model. Large steering coefficients amplify these side effects and can cause the model to enter a repetitive or incoherent generation mode where the steered direction overwhelms all other computation.

The quality of a steering vector is only as good as the contrast pairs used to construct it. If the positive and negative examples differ along dimensions other than the target concept, the vector will capture those differences as well. A "refusal" vector built from examples where the positive examples are all formal and the negative examples are all casual will encode formality along with refusal. Careful data collection and curation, using many diverse pairs that vary along the target concept while holding other dimensions roughly constant, reduces this confounding, but it requires expert judgment and is not easily automated.

A steering vector that reliably shifts behavior on a validation set of prompts may not generalize to qualitatively different prompts. If the construction dataset is narrow, the vector might only work well for prompts that are distributionally similar to the examples it was built from. This is particularly concerning for safety applications where adversarial prompts are specifically designed to be out-of-distribution.

Finally, steering changes the model's outputs but not its weights. Once the steering vector is removed, the model returns to its base behavior. This is useful for inference-time control but is not a substitute for training-based alignment when persistent behavior change is the goal. Activation steering is best understood as a complement to fine-tuning and RLHF, not a replacement.