Instruction Format › Instruction Format

When you fine-tune a language model to follow instructions, the instruction format you use to structure those instructions matters enormously. A model trained on one format will struggle to understand inputs presented in a different format at inference time. This chapter explains how templates and their special-token conventions transform raw text into structured conversations that language models can reliably interpret.

The core challenge is deceptively simple: language models process sequences of tokens, but we want them to understand the distinction between user instructions and assistant responses. We need consistent delimiters that tell the model "this is an instruction" versus "this is the expected response." These delimiters must be unambiguous, consistent during training and inference, and reliable enough to handle complex multi-turn conversations. Consistent delimiters are a prerequisite for instruction tuning to work.

Think of an instruction format as the grammar of a conversation protocol. Just as HTTP headers must follow a precise format for a web server to parse a request correctly, language model conversations must follow a precise structural convention for the model to parse roles and content reliably. The model learns this grammar from tens of thousands of training examples. If you present data in a slightly different format at inference time, you are effectively speaking a dialect the model has never studied, and the resulting degradation in output quality can be large.

As we discussed in the Instruction Following chapter, instruction-tuned models learn to recognize patterns in how instructions are presented. The format is a structural scaffold that helps the model parse inputs correctly and generate outputs in the expected positions. Without consistent formatting, the model cannot reliably distinguish between instruction and response, between the user and the assistant, or between system context and conversation content. Every structural irregularity introduces ambiguity that the model must resolve using context alone, and context alone is often insufficient.

Format design also has deeper implications for what capabilities emerge. The presence of a system-message role, for instance, makes it possible to deploy one base model across thousands of applications, each with its own persona and behavioral constraints. The way loss masking interacts with multi-turn format determines whether the model learns to generate coherent conversations or merely to predict isolated responses. These design choices shape the character of the resulting model in ways that go far beyond superficial presentation.