Self-Attention Concept › Visualizing Self-Attention

A heatmap provides an intuitive view of self-attention patterns. Rows represent query positions (which token is gathering information), and columns represent key positions (which tokens are being attended to). Darker cells indicate stronger attention.

In this random example, we see some diagonal emphasis where tokens attend to themselves, plus distributed attention to other positions. In trained models, these patterns become meaningful: related words attend strongly to each other, syntactic structures emerge, and semantic relationships become visible.

Researchers have spent considerable effort interpreting what trained attention patterns mean. In models like BERT, some attention heads learn to attend to the next token, some to the previous token, some to the sentence separator tokens. Other heads develop more linguistically interpretable patterns: there are heads that attend from verbs to subjects, from pronouns to antecedents, from nouns to modifying adjectives. This does not mean that each head has a single clean function. In practice, the patterns are messy and context-dependent. But the existence of interpretable patterns suggests that self-attention does learn to exploit syntactic and semantic structure.

The diagonal is informative. Random embeddings tend to produce some diagonal emphasis because the dot product of a vector with itself is always larger than with a random other vector. In trained models, the diagonal is often strong, meaning tokens attend significantly to themselves. This makes sense: a token's own embedding contains the most concentrated information about that token. The off-diagonal weights then represent how much additional context each token gathers from its neighbors.