Copy Mechanism › Attention Visualization for Copy

Visualizing attention patterns reveals when and what the model copies. High attention on specific source positions often indicates copying, especially when $p_{\text{gen}}$ is low.

The visual signature of copying is distinctive: when the model copies a token, attention concentrates sharply on the corresponding source position, often with more than 80% of the probability mass on a single position. When the model generates a token, attention is typically more diffuse, spread across several source positions that provide context without any single position being the "source" of the output token. This difference in attention sharpness is not just visually useful; it can be used diagnostically to understand what the model is doing at each step, and it has been used in research to study how well the model's copy decisions align with human judgments about which tokens should be copied.