Self-Attention Concept › Attention Patterns in Trained Models

While the worked example used random or hand-crafted embeddings, trained transformer models develop rich, interpretable attention patterns. Understanding what these patterns look like in practice helps build intuition for what self-attention is doing in models like BERT and GPT.

In BERT, researchers have visualized and analyzed the attention patterns of individual heads across its twelve layers. Several patterns emerge consistently. Some heads learn to attend predominantly to the next token or the previous token, effectively encoding local context. These heads are similar in function to a bigram model: they give each token information about its immediate neighbor. Other heads attend strongly to the separator tokens (the [CLS] and [SEP] tokens in BERT), which serve as aggregate representations of the entire sequence. Still other heads develop more linguistically structured patterns, with verbs attending to their subjects, determiners attending to their head nouns, and coreferent mentions attending to each other.

The emergence of these patterns is not guaranteed and is not the result of any explicit supervision signal telling the model "this head should learn dependency parsing." It arises entirely from the combination of the attention mechanism, the pretraining objective, and the optimization pressure of fitting a large training corpus. The model discovers that representing language well requires tracking syntactic and semantic relationships, and it distributes this work across its many attention heads.

In GPT-style autoregressive models, the attention patterns have a different character because causal masking prevents each position from attending to future positions. The pattern must be lower-triangular: position $i$ can only attend to positions $1, \ldots, i$. Within this constraint, heads still develop specialized functions: some heads track long-range subject-verb agreement, some attend to the most recent noun phrase (tracking the current topic), and some attend to specific function words that signal discourse structure.

In practice, the learned attention patterns are noisier and more complex than the idealized examples in textbooks. Many heads attend to a diffuse mixture of positions. This contributes to the representation without any single head being primarily responsible for a specific linguistic function. The "division of labor" across heads is more statistical than categorical. But the overall picture is clear: self-attention learns to implement something resembling the kinds of dependency structures that linguists have described, because these structures are what the training data demands.