Teacher Forcing › Data as Demonstrator

A related training paradigm called data as demonstrator (DaD) takes yet another approach. Instead of using only ground-truth tokens as teacher inputs or mixing in the model's own predictions, it trains the model by letting it observe token sequences generated by a high-quality reference model or oracle.

In the DaD framework, the process works as follows. First, a teacher model (which may be a larger, pretrained model or a well-performing domain expert) generates multiple reference sequences for each input. Then, the student model is trained to imitate the teacher's token choices. The distinction from standard teacher forcing is that the teacher's own generation errors are included, exposing the student to realistic decoder trajectories that reflect what a competent but not perfect model produces.

The advantage is that the student learns from a realistic distribution of outputs rather than from perfectly correct sequences that may never appear during inference. Human reference translations, for instance, are grammatically correct but may use different word choices than the model would naturally produce. If the model sees only human references during training, it learns to condition on human-style outputs. When it runs freely, it produces model-style outputs, creating a mismatch. A teacher model that is similar in kind to the student (a seq2seq neural network generating plausible but not perfect outputs) provides conditioning contexts that more closely resemble what the student will experience during inference.

The disadvantage is the need for a capable teacher model, which creates a bootstrapping challenge in the early stages of training. You need a good model to generate the training data for a good model. In practice, this is often handled by using a pretrained model from a related task or a previous training run, then iteratively improving both the teacher and student through multiple rounds of training.