Encoder-Decoder Framework › Limitations and Impact

The encoder-decoder framework was a landmark contribution, but it comes with several well-understood limitations that drove follow-on research.

The context vector bottleneck is the most fundamental limitation. A fixed-size vector must encode everything from a variable-length source, and the model has no mechanism to look back at specific source positions when generating a particular target token. When translating a long sentence, the decoder must rely entirely on the context vector, which may have forgotten early parts of the source by the time the encoder finishes reading.

Empirically, BLEU scores for basic seq2seq models drop noticeably on sentences longer than 20 to 30 tokens. The encoder's hidden state at position $T$ has strong memory of tokens near the end of the sequence but weak memory of tokens near the beginning, due to the way gradients vanish over time in RNNs. The attention mechanism, introduced by Bahdanau et al. in 2015, directly addresses this by giving the decoder soft access to all encoder hidden states at every decoding step.

The magnitude of the improvement from attention is substantial. On the WMT English-French benchmark, adding attention to the basic seq2seq model improved BLEU by roughly 3 to 5 points, closing much of the gap between the basic neural model and the then-best phrase-based systems. More importantly, the attention-equipped model's performance no longer degraded sharply with sentence length: it maintained near-constant BLEU across the full range of sentence lengths, while the attention-less model deteriorated rapidly beyond 20 tokens.

Teacher forcing creates a training-inference mismatch. During training, the decoder always receives correct context, the ground-truth previous tokens. During inference, it receives its own predictions, which may be wrong. Errors compound: a mistake at position 5 creates wrong context for positions 6, 7, 8, and so on. This exposure bias becomes especially harmful in long sequences.

Scheduled sampling partially mitigates this, at the cost of more complex training dynamics. The Teacher Forcing chapter explores these tradeoffs in detail.

Basic seq2seq uses greedy decoding at inference: the decoder always picks the highest-probability token at each step. This is fast but suboptimal because a locally optimal sequence of token choices does not necessarily maximize the total sequence probability. Beam search maintains multiple candidate hypotheses simultaneously and selects the globally best sequence, achieving higher BLEU scores at increased computational cost. The Beam Search chapter covers this.

Because the LSTM encoder processes tokens one at a time, left to right, the encoder computation cannot be parallelized across the sequence. Each hidden state depends on the previous one. For long sequences or large models, this sequential dependency creates a bottleneck in information capacity and in wall-clock training time. The transformer architecture addresses this by replacing recurrence with self-attention, which can process all positions in parallel. As we will see in later chapters, this parallelism was arguably the most important practical advantage of transformers over seq2seq, enabling training on orders of magnitude more data.

Despite these limitations, the encoder-decoder framework changed neural sequence modeling. Before seq2seq, neural machine translation systems required careful alignment models, phrase tables, and language models assembled by hand. The seq2seq model replaced all of that with a single, end-to-end trained neural network. The 2014 papers by Sutskever et al. and Cho et al. demonstrated competitive or superior performance on English-French translation compared to state-of-the-art phrase-based systems, purely from parallel sentence data.

The framework also opened up a vast range of new applications. Summarization, dialogue, code generation, image captioning, speech recognition, and question answering all became tractable with the same basic architecture. The combination of encoder-decoder with attention became the standard for all sequence transformation tasks, and the attention mechanism within this framework eventually inspired the transformer architecture that dominates NLP today. The original transformer paper, "Attention is All You Need," replaced the recurrent encoder and decoder with stacked self-attention layers, but kept the fundamental encoder-decoder split intact. The encoder still produces a sequence of representations from the source, and the decoder still generates the target autoregressively, consulting the encoder outputs through cross-attention at each step.

Virtually every modern language model is a descendant of the encoder-decoder insight. BERT is an encoder trained on masked language modeling. GPT is a decoder trained on causal language modeling. T5 and BART are full encoder-decoder models. The specific architectures have evolved enormously, but the underlying intuition, that reading and generating are different tasks that benefit from different architectures connected through a learned representation, remains at the heart of the field.