Attention Intuition › Attention Intuition

In the encoder-decoder framework we covered in the Encoder-Decoder Framework chapter, the model compresses an entire input sequence into a single fixed-length vector. This works reasonably well for short sequences, but as inputs grow longer, that single vector becomes an information bottleneck. How can we expect one vector to faithfully represent a 50-word sentence, let alone an entire paragraph? The encoder's final hidden state must somehow encode every word, every phrase, every syntactic relationship, and all of this must fit into a vector of fixed size. The model is asked to perform an impossible compression: pour a gallon of meaning into a pint-sized vector.

Attention solves this by allowing the decoder to look back at all encoder states and decide which parts of the input are most relevant at each decoding step. Instead of relying on a compressed summary, the model learns to focus on different parts of the input as it generates each output token. This simple idea dramatically improved sequence-to-sequence models and became the foundation for the transformer architecture at the heart of modern NLP.

This chapter builds the intuition for how attention works: why it is needed, what it computes, how to interpret the results, and how different formulations compare. You will come away with a thorough understanding of attention as a general mechanism before we examine Bahdanau attention and Luong attention as concrete implementations in the following chapters.