Activation Steering: Steering Vectors and Representation Engineering › Summary

Activation steering provides a principled way to modify language model behavior at inference time by injecting directional vectors into the model's residual stream. The technique rests on the linear representation hypothesis: many concepts are encoded as directions in activation space, making them addable and subtractable through simple vector arithmetic.

Steering vectors are constructed by computing the mean activation difference between examples of a concept and examples of its contrast. The contrastive activation difference method averages activations over positive and negative prompt sets. Paired prompts reduce noise by matching example pairs that differ only in the target concept. PCA-based methods extract the dominant direction of variation from a more diverse set of examples.

Once constructed, a steering vector is applied by registering a forward hook that adds a scaled version of the vector to the residual stream at a target layer. The scaling coefficient $\alpha$ controls intervention strength: positive values push toward the concept, negative values push away. Normalizing the vector to unit length makes $\alpha$ interpretable as a distance in activation space.

The representation engineering framework extends activation addition into a systematic approach with explicit representation reading (measuring internal states with a reading vector) and representation writing (modifying them with a control vector), enabling more controlled and verifiable interventions. The separation between reading and writing allows verification that the vector is capturing the intended concept before it is used for control.

Key diagnostics include the layer-by-layer steering vector norm (to identify the optimal target layer), the cosine similarity between steering directions (to check for entanglement between concepts), and the next-token probability comparison before and after steering (to rapidly verify that the vector captures a real semantic direction).

Applications span reducing harmful outputs, improving honesty and reducing sycophancy, enabling real-time jailbreak monitoring, and testing causal hypotheses about how internal representations drive behavior. Limitations include dependence on the linearity of representation, sensitivity to layer and coefficient choice, side effects on related concepts, and lack of persistence once steering is removed.

As we move into the next part of this book, the ability to both read and write model representations connects activation steering to the broader challenge of reliably aligning language model behavior with human intentions. Activation steering is not the final answer to alignment, but it is one of the most direct and interpretable tools we currently have for making the connection between internal representation and external behavior explicit and manipulable.