Teacher Forcing › Comparing Training Strategies

Each training strategy makes different tradeoffs between training stability, inference quality, and computational cost. Understanding these tradeoffs helps you choose the right approach for a given task and budget.

Pure teacher forcing is the most common approach and the default for most seq2seq systems. It provides fast training with stable gradients because every training step conditions on correct context. Performance with teacher-forced evaluation is strong, but there is a significant exposure bias at inference time for long sequences. For many tasks, especially with modern attention-based models, the exposure bias is manageable and teacher forcing alone produces competitive results.

Free-running training (no teacher forcing from the start) is practically infeasible for most tasks because of cascading errors. The training signal is noisy and unstable, and convergence either fails or is extremely slow. Free-running training is not used as a primary training approach; it is primarily a theoretical baseline that illustrates why teacher forcing was needed in the first place.

Scheduled sampling represents a practical middle ground. Training speed is intermediate between teacher forcing and free-running, and the model progressively learns to handle its own outputs. The schedule hyperparameters require tuning, and the inconsistency from mixed conditioning contexts can sometimes slow learning or require more model capacity. But for tasks with significant exposure bias issues, scheduled sampling reliably reduces the train-test gap.

REINFORCE and SCST directly optimize the task metric, which is their primary advantage. But they suffer from high gradient variance, require sampling complete sequences during training (computationally expensive), and can be unstable without careful hyperparameter management. These approaches are most commonly used as a fine-tuning step after teacher forcing pretraining, not as the primary training method.

In practice, the choice of training strategy also depends on the architecture. Transformer-based seq2seq models tend to show less exposure bias than RNN-based models, probably because their attention mechanisms allow them to condition flexibly on all previous positions rather than relying solely on a compressed hidden state that may carry errors. For transformer models, pure teacher forcing often suffices. For RNN models on long sequences, scheduled sampling or RL fine-tuning may be necessary to achieve good free-running performance.