Teacher Forcing › Limitations and Practical Impact

Teacher forcing was a critical enabler for practical seq2seq training, but it comes with real tradeoffs that are worth understanding deeply before applying it.

The exposure bias problem has measurable practical effects. In neural machine translation benchmarks, models trained with pure teacher forcing often show a significant drop in BLEU score when the source sentences are longer or when beam search is used with a larger beam width. The model, having never encountered its own errors during training, lacks the learned ability to course-correct when decoding goes off track. This effect is most pronounced for languages with long-range dependencies or flexible word order, where an error early in generation can propagate for many tokens before the sentence structure constrains the output back toward coherence.

The gap between teacher-forced training performance and free-running inference performance led researchers in the 2015-2018 period to develop the mitigation strategies described in this chapter. Scheduled sampling reduced the gap but did not eliminate it. Professor forcing added a representation-level constraint but required adversarial training with all its instabilities. REINFORCE-based approaches directly optimized task metrics but were expensive and noisy. None of these approaches fully solved the problem; each traded one set of difficulties for another.

Scheduled sampling alleviates exposure bias but does not fully resolve it. The mixing of teacher-forced and free-running tokens within the same training sequence creates a hybrid conditioning context that can confuse the model. Theoretically, the model should learn to handle both modes, but in practice the inconsistency sometimes hurts learning efficiency. Some researchers have found that scheduled sampling requires careful hyperparameter tuning and can be sensitive to the choice of schedule. The inconsistency problem is particularly sharp at the boundary between teacher-forced and free-running positions: a sequence where positions 1-4 are teacher-forced and position 5 is free-running presents the model with an unusual conditioning context that it would never encounter during pure inference.

A practical observation from industrial NMT systems is that even with exposure bias, teacher forcing produces competitive models when paired with beam search at inference time. Beam search effectively hedges against early mistakes by maintaining multiple hypotheses simultaneously, re-evaluating them as the sequence grows and discarding branches that fall too far behind the leading candidates. The next chapter covers beam search in detail and explores how its width and length normalization interact with the training strategy. The interaction is significant: models trained with teacher forcing tend to respond well to beam search because they have learned sharp conditional distributions over correct sequences, and beam search can exploit these sharp distributions to find good completions even when early steps are uncertain.

The advent of transformer-based seq2seq models has somewhat shifted the conversation about exposure bias. Transformers trained with masked self-attention use the same teacher forcing paradigm, and they have proven substantially more robust to exposure bias than RNN-based models. This robustness likely comes from several factors. Their attention mechanisms allow them to condition on all previous positions simultaneously, which reduces the compounding error effect: a wrong token at position $t$ propagates its influence through attention weights, but the weights to other correct tokens can compensate. Their large capacity allows them to learn more robust conditional distributions that generalize better across the small distribution shift from teacher-forced to free-running contexts. And their parallel training (which requires teacher forcing) allows them to be trained at scales where the sheer volume of diverse training examples reduces the impact of any particular conditioning pattern being out-of-distribution at inference time.

Nevertheless, the fundamental train-test mismatch remains in all teacher-forced models, transformer or otherwise. Understanding it is essential for diagnosing generation failures. When a model produces incoherent long-form text despite good perplexity on held-out data, exposure bias is often the culprit. When a model succeeds on short test inputs but fails on longer ones, the compounding nature of exposure bias is a natural suspect. When RL fine-tuning dramatically improves generation quality without changing the model's perplexity, it is often because the RL phase is teaching the model to handle the imperfect contexts it will encounter at inference time.

For practitioners building production sequence generation systems, the most common recipe is to train with pure teacher forcing to convergence, then apply a brief RL fine-tuning phase using a mixed cross-entropy and task-metric objective. This strategy combines the efficiency of teacher forcing with the inference-time optimization of reinforcement learning, and it has been validated across machine translation, abstractive summarization, image captioning, and dialogue systems.