Teacher Forcing › Professor Forcing: A Different Angle

An alternative approach, proposed by Lamb et al. (2016), is professor forcing. Rather than mixing teacher-forced and free-running training at the input level, professor forcing uses an adversarial training objective to align the model's hidden state distributions between the two modes.

The approach trains a discriminator to distinguish between two types of hidden state trajectories. The first type consists of hidden states produced by teacher forcing, where inputs are ground-truth tokens and the decoder's internal representations reflect a clean, correct sequence of states. The second type consists of hidden states produced by free-running decoding, where inputs are the model's own predictions and the representations reflect the actual trajectory the model would follow at inference time.

The generator, which is the seq2seq model itself, then tries to fool the discriminator by producing hidden states under free-running conditions that look statistically indistinguishable from teacher-forced hidden states. This adversarial signal pushes the free-running trajectory to stay close to the teacher-forced trajectory in representation space, not just in output token space.

Why is this a different and potentially more powerful approach than scheduled sampling? Scheduled sampling addresses the problem at the input level: it teaches the model to produce reasonable outputs when given its own previous output as input. But it does not directly constrain how the model's hidden states evolve during free-running. The hidden states are what the model uses to compute its output distributions, so if the hidden states under free-running diverge substantially from those under teacher forcing, the model's behavior will still degrade even if the individual input tokens are similar.

Professor forcing adds an explicit objective that the model's internal representations should remain consistent regardless of whether it is operating in teacher-forced or free-running mode. This forces a kind of representation-level robustness: the model must learn to maintain useful hidden state structure even when its inputs are its own potentially imperfect predictions. The discriminator provides a direct signal about whether the internal states are deviating from the teacher-forced distribution, which is something that cross-entropy loss alone cannot measure.

The practical limitation of professor forcing is that it requires training an adversarial discriminator alongside the main model. Adversarial training is notoriously sensitive to hyperparameter settings and can be difficult to stabilize. The discriminator must be capable enough to detect distribution differences but not so powerful that it makes the generator's task impossible. Despite these challenges, professor forcing demonstrated that addressing exposure bias at the representation level rather than just the input level is a valid and effective approach, and it inspired later work on representation regularization for sequence models.