Query, Key, Value › Summary

Query, Key, Value projections transform self-attention from a fixed similarity computation into a learned matching mechanism. By giving each token separate representations for different roles, the model can learn arbitrary attention patterns that capture linguistic relationships.

Key takeaways from this chapter:

• Three roles, one mechanism: Queries express what a token seeks, keys advertise what a token offers, and values provide what gets aggregated. This separation enables flexible, learned attention patterns.

• Projection matrices are learned: $\mathbf{W}_Q$, $\mathbf{W}_K$, and $\mathbf{W}_V$ are the trainable parameters. During training, they discover which aspects of embeddings matter for matching and aggregation.

• Query-key matching determines attention: The dot product $\mathbf{Q}\mathbf{K}^T$ computes compatibility scores. High scores mean a query matches a key well, leading to high attention weight.

• Scaling prevents gradient issues: Dividing by $\sqrt{d_k}$ keeps score magnitudes stable regardless of dimension. This ensures softmax operates in a regime with healthy gradients.

• Values carry the content: Attention weights determine how much each position contributes, but value vectors determine what content flows. A token can have a key that attracts attention while its value transmits different information.

• Dimensions have meaning: Query and key dimensions must match ($d_k$) for dot product compatibility. Value dimension ($d_v$) determines output size. In practice, $d_k = d_v$ for simplicity.

In the next chapter, we'll explore multi-head attention, which runs multiple QKV attention operations in parallel. This allows the model to capture different types of relationships simultaneously, dramatically increasing the expressiveness of the attention mechanism.