Instruction Tuning Training › Summary

Instruction tuning training requires balancing multiple competing concerns: so adequate coverage of minority tasks without overfitting, focusing learning on response generation through loss masking, and selecting hyperparameters that preserve pre-trained capabilities while teaching instruction following.

The three major decisions we covered in this chapter work together as an interconnected system:

1. Data mixing strategy: Use temperature-based sampling with $T \approx 2$ to $3$ to balance task representation. This ensures minority tasks receive enough training signal while common tasks remain appropriately frequent. Cap repetition of small datasets at 3 to 5 repeats to prevent memorization.
2. Loss masking: Zero out the loss on all prompt tokens so that every gradient update comes from response generation. This is not optional for instruction tuning to work well. Without it, the model spends significant capacity on pre-training-style text prediction rather than instruction following. Remember to normalize by the number of active (unmasked) tokens, not by sequence length.
3. Hyperparameter selection: Use lower learning rates ($1$ to $5 \times 10^{-5}$) than pre-training to avoid catastrophic forgetting. Use a short warmup period (about 3% of training steps) to stabilize early updates. Train for 1 to 3 epochs and monitor validation loss on a diverse held-out set, using early stopping when validation loss begins to rise. Target effective batch sizes of 128 to 256 through gradient accumulation if memory is limited.

These techniques work together to produce models that follow instructions reliably across diverse task types while maintaining their broader language capabilities. The next chapter examines how to evaluate whether instruction tuning has succeeded, moving from the training mechanics we covered here to the equally important question of measuring the resulting model's behavior.