Copy Mechanism › Copy Mechanism

Standard sequence-to-sequence models generate output tokens by selecting from a fixed vocabulary. This works well for common words, but what happens when the input contains a rare proper name, a technical term, or a number that the model has never seen? The decoder must either hallucinate a substitute or produce a generic placeholder. Neither outcome is satisfactory.

Copy mechanisms solve this problem by allowing the decoder to directly copy tokens from the input sequence. Instead of being forced to generate every output token from scratch, the model can "point" to input positions and reproduce those tokens verbatim. This matters for tasks like summarization, where preserving names and facts is critical, and for handling out-of-vocabulary words that would otherwise be lost.

Think of it like the difference between a journalist writing from memory versus one with the original press release in hand. The journalist writing from memory must rely on their prior knowledge, which may not include the exact spelling of a foreign diplomat's name, a precise dollar figure, or a specific product model number. The journalist with the source document in front of them can simply copy those details verbatim while still composing their own narrative around them. Copy mechanisms give neural models that same advantage: access to the source text as a direct lookup resource, not just as a context to be encoded and compressed into a fixed hidden state.

The challenge is architectural. A standard decoder outputs a probability distribution over a fixed vocabulary at each step. Names like "Nakamura" or "Pfizer" or numbers like "47.3%" may simply not exist in that vocabulary, meaning no matter how much the model attends to them in the encoder, it has no mechanism to reproduce them. The copy mechanism threads a new pathway from the attention distribution directly to the output, bypassing the vocabulary bottleneck for tokens that are better reproduced than generated.

This chapter explores how copy mechanisms work, from the foundational pointer networks to the practical pointer-generator architecture. You'll learn how models compute copy probabilities, blend copying with generation, and handle the challenging case of out-of-vocabulary tokens. By the end, you'll understand why copy mechanisms became a standard component in production summarization systems and how they relate to the broader challenge of faithfulness in neural text generation.