Copy Mechanism › Practical Training Considerations
Language AI HandbookAll chapters Copy Mechanism › Practical Training ConsiderationsOne section · Sections in this chapter11 / 14 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Training pointer-generator networks requires careful attention to several practical details. The architecture introduces several interacting components, each with its own training dynamics, and getting them to work together reliably requires understanding both the mechanics and the common failure modes.
Teacher forcing with copy targets: During training, when the target token appears in the source, the model should learn to copy it. This requires computing whether each target token is copyable and adjusting the loss accordingly. The standard cross-entropy loss applies to the extended vocabulary distribution: if the target is an OOV token that appears in the source, the model should learn to assign high probability to its extended vocabulary ID, which it can only do through the copy channel. This implicitly trains the copy mechanism to activate for OOV tokens without requiring explicit copy labels.
def prepare_copy_targets(source_ids, target_ids, vocab_size):
"""
Prepare targets for copy-aware training.
Returns mask indicating which target tokens are copyable from source.
"""
batch_size, tgt_len = target_ids.shape
_, src_len = source_ids.shape
# For each target position, check if token appears in source
copyable = torch.zeros(batch_size, tgt_len, dtype=torch.bool)
copy_positions = torch.zeros(batch_size, tgt_len, src_len)
for b in range(batch_size):
for t in range(tgt_len):
tgt_token = target_ids[b, t].item()
for s in range(src_len):
if source_ids[b, s].item() == tgt_token:
copyable[b, t] = True
copy_positions[b, t, s] = 1.0
# Normalize copy positions
copy_positions = copy_positions / (
copy_positions.sum(dim=-1, keepdim=True) + 1e-12
)
return copyable, copy_positions
# Example
source_ids = torch.tensor(
[[5, 8, 9, 4, 10]]
) # "the Chen Stanford said discovery"
target_ids = torch.tensor([[8, 11, 10]]) # "Chen announced discovery"
copyable, copy_pos = prepare_copy_targets(source_ids, target_ids, vocab_size=20)Copy Target Preparation
==================================================
Source IDs: [5, 8, 9, 4, 10]
Target IDs: [8, 11, 10]
Target token analysis:
Chen: copyable
Copy from source position(s): [1]
announced: must generate
discovery: copyable
Copy from source position(s): [4]The prepare_copy_targets function above computes two useful outputs. The copyable mask identifies which target positions can be satisfied through copying, which is useful for analysis and debugging. The copy_positions tensor identifies exactly which source positions contain each target token, which can be used to compute an auxiliary "copy alignment" loss that explicitly trains the attention mechanism to concentrate on the right source positions when copying. This auxiliary loss is optional but can speed up early training when the copy mechanism is still learning to associate target tokens with their source positions.
Additional training techniques include:
- Scheduled sampling for : Early in training, the model may not learn when to copy effectively. Some implementations initialize with a bias toward copying for OOV tokens, then let learn freely once the basic copy mechanism is working.
- Gradient clipping: The combined distribution can have very small probabilities for OOV tokens early in training, leading to large gradients through the loss. Gradient clipping at a norm of 2.0 to 5.0 helps stabilize training during the early phases.
- Coverage loss weighting: The coverage loss is added to the main cross-entropy loss with a weighting coefficient, typically tuned between 0.5 and 2.0. Too high a weight makes the model too aggressive at avoiding repetition, which can hurt fluency. Too low a weight and the coverage mechanism has insufficient effect. A common practice is to start with no coverage loss, train until convergence, then fine-tune with coverage loss added.
- Batch construction: Since extended vocabulary sizes vary with the source document length and OOV count, batching requires padding both the source sequences and the extended vocabulary distributions to a uniform size. A maximum OOV count per batch (typically 50 to 200) prevents the extended vocabulary from growing excessively large for batches containing very long or vocabulary-diverse documents.
The boxplot reveals the learned specialization that emerges from training without any explicit annotation of which tokens should be copied. Function words like "the," "of," and "in" cluster near 1.0 because the model learns they are never worth copying: they are far more reliably generated from the vocabulary than pointed to in the source. Numbers and named entities cluster near 0.0 because the model learns that whenever they appear in the target, the safest strategy is to copy them directly from the source. Content words occupy the most interesting middle ground, where context determines the optimal strategy and the distribution is correspondingly wide.