Teacher Forcing › Scheduled Sampling: Bridging the Gap

Bengio et al. (2015) proposed scheduled sampling as a curriculum learning approach to address exposure bias. The core idea is to gradually transition from teacher forcing to free-running training during the training process, so the model learns to handle its own outputs before deployment.

In scheduled sampling, at each training step you flip a biased coin. With probability $\epsilon_t$ (the "scheduled" probability), you feed the true previous token as input (teacher forcing mode). With probability $1 - \epsilon_t$, you feed the model's own previous prediction (free-running mode).

$$\text{input}_t = \begin{cases} y_{t-1} & \text{with probability } \epsilon_t \\ \hat{y}_{t-1} & \text{with probability } 1 - \epsilon_t \end{cases}$$

where:

• $\epsilon_t$: the teacher forcing probability at training step $t$, which decays over time
• $y_{t-1}$: the ground-truth token at position $t-1$
• $\hat{y}_{t-1}$: the model's own predicted token (argmax or sampled) at position $t-1$

The scheduled part comes from how $\epsilon_t$ is decayed over training. Three common schedules are:

Linear decay: $\epsilon_i = \max(\epsilon_{\min}, k - i \cdot c)$, where $i$ is the training iteration and $c$ controls decay speed. This reduces teacher forcing at a constant rate, giving a predictable and easy-to-tune schedule. The $\epsilon_{\min}$ floor prevents teacher forcing from going all the way to zero, which helps maintain some training stability even late in training.

Exponential decay: $\epsilon_i = k^i$, where $k < 1$. This decays quickly at first, then slows as it approaches zero. Because the schedule starts steep, the model is transitioned away from teacher forcing aggressively in early iterations and then receives a more gradual nudge in later iterations. This is suitable when you want the model to encounter its own errors early in training.

Inverse sigmoid decay: $\epsilon_i = k / (k + \exp(i / k))$. This produces an S-shaped schedule, staying near 1.0 for a while before declining steadily to near 0.0. The effect is that the model receives nearly full teacher forcing during its early learning phase, which is when the basic conditional distributions are being established. Then, once the model has developed some baseline capability, the schedule allows the teacher forcing ratio to drop, progressively exposing the model to its own outputs.

The intuition behind the curriculum approach is that a model needs to first learn the basic conditional distributions (what tokens follow what) before it can learn to recover from mistakes. Starting with pure teacher forcing provides stable gradients for this initial learning. Then, as training progresses, introducing the model's own outputs teaches it to remain coherent even when its context is imperfect.

The choice of schedule affects more than just training speed. A schedule that drops $\epsilon$ too quickly will destabilize training before the model has enough capability to learn from free-running feedback. A schedule that drops $\epsilon$ too slowly may not provide enough exposure to the model's own errors before training ends, leaving significant residual exposure bias. In practice, the inverse sigmoid schedule is often recommended because it provides the best of both regimes: a stable initial phase and a controlled transition.