Teacher Forcing › How Teacher Forcing Works in Practice

The mechanics of teacher forcing are straightforward once you see the decoder's input-output structure clearly.

In an encoder-decoder model, the decoder receives two inputs at each time step. The first is the previous token embedding, which is the recurrent input carrying information about what was generated just before. The second is the encoder's context information, which is either a single fixed hidden state passed from the encoder or, in models with attention, a dynamically weighted combination of encoder states. Teacher forcing changes only the first of these: how the "previous token" is determined.

With teacher forcing, the "previous token" input is always taken from the ground-truth target sequence, shifted right by one position. If the target sequence is $\langle\text{SOS}\rangle, y_1, y_2, \ldots, y_T, \langle\text{EOS}\rangle$, then:

• At step 1: decoder receives $\langle\text{SOS}\rangle$ as input, should predict $y_1$
• At step 2: decoder receives $y_1$ as input, should predict $y_2$
• At step $t$: decoder receives $y_{t-1}$ as input, should predict $y_t$
• At step $T+1$: decoder receives $y_T$ as input, should predict $\langle\text{EOS}\rangle$

The key insight is that the decoder sees the entire target sequence shifted one position to the right, so every conditional probability is conditioned on the true context rather than noisy predictions. In practice this is efficient to implement: the entire target sequence can be fed into the decoder in parallel (for transformer architectures) or in a tight loop where each teacher-forced input is a tensor slice from a pre-loaded batch of targets.

It is worth pausing on why this works so efficiently from a computational perspective. Because the teacher-forced inputs are all drawn from the target sequence (known ahead of time), you can precompute the entire sequence of decoder inputs before starting the forward pass. In RNN-based seq2seq models, this allows a tight training loop. In transformer-based models, it allows fully parallel processing of all decoder positions simultaneously, which is a key reason transformers can be trained so much faster than RNNs on modern hardware. Teacher forcing improves accuracy and is also a prerequisite for the computational efficiency that makes large-scale training feasible.

Transformers exploit teacher forcing more aggressively than RNNs. In an RNN decoder, even with teacher forcing, you must step through positions sequentially because each hidden state depends on the previous one. In a transformer decoder, the entire target sequence is fed in at once, and the attention mechanism computes all positions in parallel. The causal masking ensures that position $t$ can only attend to positions $1, \ldots, t-1$, which enforces the autoregressive property while preserving full parallelism during training.

This means transformer training processes all $T$ tokens simultaneously per layer pass, rather than requiring $T$ sequential RNN steps. The result is the dramatic training speedup that made models like BERT and GPT practical at scale. Teacher forcing is the mechanism that makes this parallel training coherent: without it, you would need to wait for each prediction before feeding the next input, collapsing back to sequential processing.

The consequence is that teacher forcing is architecturally necessary for the transformer's parallel training regime. You cannot train a transformer decoder without providing it the full right-shifted target sequence as input during the forward pass. The causal masking handles the autoregressive constraint, and the teacher-forced targets handle the input. Remove either of these and the transformer's training paradigm breaks down.