Teacher Forcing › Curriculum Learning and the Training Philosophy

Scheduled sampling is an instance of a broader idea called curriculum learning, where the training distribution is intentionally structured to present easier examples first, then progressively harder ones. The "easy" version here is teacher forcing, where the model never needs to recover from mistakes. The "hard" version is free-running decoding, where the model must handle all the consequences of its own imperfect outputs.

This curriculum approach mirrors how humans learn to perform complex sequential tasks. A student learning to write learns first by copying model sentences, which is analogous to teacher forcing. Then they move to writing with occasional corrections, where the teacher steps in when errors would compound badly. Finally they write independently, handling their own mistakes as they arise. Direct exposure to the fully independent task from the start would be overwhelming because the feedback signal is too noisy to learn from efficiently; the curriculum manages the complexity gradient.

Curriculum learning in general has been shown to accelerate training across many domains beyond seq2seq models. The key principle is that the learning signal should be appropriately calibrated to the student's current capability. When the student is weak, easy examples with clean feedback are most informative. When the student is stronger, harder examples that push the boundaries of what they can handle accelerate further learning. Scheduled sampling applies this principle specifically to the question of input quality in sequential decoding.

One subtle point is that scheduled sampling is not a perfect solution. It can introduce inconsistency: at any given training step, some positions in the sequence use teacher forcing while others use free-running predictions. This creates a hybrid conditioning context that the model must handle, which can sometimes make learning harder rather than easier. The model sees sequences where some tokens are correct (from the ground truth) and adjacent tokens are potentially wrong (from the model's own predictions), creating conditioning contexts that are neither fully correct nor fully free-running. Handling this mixture requires the model to develop a kind of robustness to partial context errors, which may require more model capacity or more careful training management.

A further subtlety arises when you consider gradient computation. With pure teacher forcing, gradients flow cleanly through fixed inputs. With scheduled sampling, the model's own predictions are used as inputs, and those predictions are the output of differentiable operations. In principle, you could backpropagate through the sampling operation itself to train the model end-to-end. In practice, most implementations treat the sampled tokens as fixed and do not backpropagate through the sampling step, because the argmax (or categorical sample) is not differentiable. This means scheduled sampling does not fully exploit the information available in the model's own predictions, leaving some optimization potential on the table.