Teacher Forcing › The Exposure Bias Problem

Teacher forcing accelerates training significantly, but it introduces a critical problem: the model is trained on a distribution that it never encounters during inference.

During training with teacher forcing, the decoder always receives correct tokens as input. The model learns the conditional distribution $P(y_t \mid y_1, \ldots, y_{t-1}, \mathbf{c})$ where the conditioning context is always drawn from the true sequence.

During inference (free-running decoding), the decoder receives its own previous predictions. The model encounters the distribution $P(y_t \mid \hat{y}_1, \ldots, \hat{y}_{t-1}, \mathbf{c})$ where the context consists of potentially imperfect predictions.

This mismatch is called exposure bias because the model is never "exposed" to its own errors during training. The consequence is that even small mistakes early in generation can derail the model: the first wrong token puts the decoder in a state it never encountered during training, which leads to another wrong token, which leads to another unfamiliar state, and so on. This compounding of errors is sometimes called the "snowball effect."

To develop strong intuition for why this matters, consider a concrete example. Suppose you are training a translation model, and in the target language, a particular grammatical construction requires a specific word order. The model has seen thousands of examples of this construction during training, always with perfect teacher-forced context. It has learned to produce the correct word order given correct input.

At inference time, the model generates the first word of this construction correctly, but then makes a small error in the second word. Now the third word must follow a malformed second word, a situation the model has never seen during training. Because this state is out-of-distribution, the model has no reliable behavior here. It might produce something locally plausible given the wrong second word, pushing the output further in the wrong direction, or it might generate something entirely unexpected because this input region is sparse in the model's learned distribution.

The severity of exposure bias depends on sequence length and output vocabulary size. Short sequences with limited vocabulary (like simple arithmetic operations) may show minimal degradation. Long sequences with large vocabularies (like paragraph-length text generation) can suffer substantially: a model that achieves near-perfect performance with teacher forcing might generate incoherent text when running freely. This explains why early seq2seq models often performed well on benchmark metrics computed with teacher-forced decoding but showed noticeably worse quality in real deployment, where no teacher is present.

To quantify the gap formally, define the decoder's true data distribution at step $t$ as $p_{\text{data}}(y_t \mid y_{<t})$ and the model's prediction as $p_{\theta}(y_t \mid y_{<t})$. At training step $t$, the input is $y_{<t} \sim p_{\text{data}}$ (teacher forcing), but at inference step $t$, the input is $\hat{y}_{<t} \sim p_{\theta}$. As sequences grow longer, $\hat{y}_{<t}$ diverges increasingly from $y_{<t}$, amplifying the distribution shift. The compounding nature of this divergence is what makes exposure bias worse for longer sequences: each step's imperfection contributes to pushing the inference-time trajectory further from the training-time trajectory.

One way to measure exposure bias directly is to evaluate a model using two different decoding strategies and compare. The first strategy is teacher-forced decoding, where even at "inference time" you provide correct previous tokens. This measures the model's upper-bound performance in the perfect-information setting. The second strategy is free-running decoding, where the model must use its own previous predictions. The gap between these two metrics quantifies the severity of exposure bias for that model.

Large gaps indicate the model has learned very precisely calibrated conditionals over correct sequences but struggles to generalize to imperfect sequences. Smaller gaps suggest the model is either more robust to distributional shift, or that the task itself has less compounding error because the vocabulary is small or sequences are short. Monitoring this gap during training can tell you whether a particular mitigation strategy is working, and whether you need to invest in more aggressive exposure bias reduction.