Instruction Format › Prompt Templates

A prompt template defines the structure that wraps each instruction-response pair during training. Think of it as a contract between you and the model: you promise to always present instructions in a certain way, and in return, the model learns to recognize and respond to that presentation reliably. At its simplest, a template specifies where the instruction goes, where the response goes, and what separates them. However, template design has a significant impact on model behavior, affecting everything from how confidently the model generates responses to how gracefully it handles edge cases in multi-turn dialogue.

Language models are pattern recognition engines. During pre-training, they learn statistical regularities in text. During instruction tuning, we want them to learn a new regularity: the pattern that connects instructions to appropriate responses. The template creates this pattern by giving consistent structural cues that the model can latch onto. Without these cues, the model would have no way to determine where an instruction ends and where it should begin generating. The template is the primary mechanism through which the model learns the concept of "following an instruction."

Consider the most basic template structure:

This template uses natural language markers to delineate sections. The model learns that text after ### Instruction: contains what it should respond to, and text after ### Response: contains the expected output. During training, both sections are present. This allows the model to learn the mapping from instruction to response. During inference, you give everything up through ### Response: and let the model generate the continuation. The blank space after ### Response: is an invitation for the model to fill in what should come next.

Why do these markers work? The model learns through thousands of training examples that whenever it sees ### Instruction: followed by some text and then ### Response:, the text that follows ### Response: should be a helpful, relevant answer to whatever appeared after ### Instruction:. The consistency of this pattern allows the model to generalize: when it encounters the same structure with a new instruction at inference time, it knows exactly what role it should play. This generalization is the foundation of instruction tuning, and it depends entirely on the template being applied with perfect consistency throughout the training process.

The choice of marker matters more than it might initially appear. Natural language markers like ### Instruction: have the advantage of being human-readable and easy to work with during development and debugging. However, they carry a risk: the text of the marker might appear in normal content. Imagine a training example that asks the model to explain what the Alpaca ### Instruction: format is. The model would encounter ### Instruction: embedded within what is supposed to be instruction content, creating an ambiguous signal about where one section ends and another begins. This is why the community progressively shifted toward special tokens.

The Alpaca dataset, which pioneered many instruction-tuning practices, used a slightly more elaborate template that included an optional input field:

This three-part structure separates the task description from any context needed to complete it. For a translation task, the instruction might be "Translate the following text to French" while the input contains the actual text to translate. This separation helps the model distinguish between instructions and context. The preamble at the beginning also serves an important function: it primes the model with context about what kind of task this is, setting expectations before the specific instruction appears. The model processes that preamble and adjusts its attention accordingly, arriving at the specific instruction already calibrated to produce a "response that appropriately completes the request."

When the input field is empty, the Alpaca format omitted it entirely and used a different preamble: "Below is an instruction that describes a task. Write a response that appropriately completes the request." This variability in the preamble means the model learns that the preamble is informative but not a hard structural signal; the real markers are ### Instruction: and ### Response:. Recognizing these as the primary anchors helps the model generalize across both preamble variants.

Modern templates typically use special tokens rather than natural language markers. Special tokens are added to the tokenizer's vocabulary specifically to serve as structural delimiters. They are less likely to appear in normal text, making them unambiguous markers of format structure. Consider the difference: if you use ### Response: as a marker, and you ask the model to "explain what ### Response: means in the Alpaca format," the model might become confused about where the actual response should begin. Special tokens like <|assistant|> avoid this ambiguity because they exist outside the normal vocabulary of human-written text. The tokenizer always maps them to a single, dedicated token ID that cannot be produced by composing subword pieces from ordinary text.

The transition from natural-language markers to special tokens also changed how models stand for format structure internally. A natural-language marker like ### Response: is tokenized into a sequence of tokens that each carry semantic meaning from pre-training. The hash characters, the word "Response," the colon, all carry prior statistical associations. A special token like <|im_end|> starts as a blank slate during fine-tuning; its entire learned meaning derives from its structural role in the conversation format. This gives the model a cleaner signal: the special token says "structural boundary" without any confounding semantic content.

Let's trace a concrete instruction-response pair through the Alpaca template step by step, tracking exactly what the model sees and what it learns to predict.

Raw training pair:

• Instruction: "Summarize the following paragraph in one sentence."
• Input: "The Amazon rainforest, often called the 'lungs of the Earth,' produces 20% of the world's oxygen and is home to 10% of all species."
• Response: "The Amazon rainforest produces a fifth of global oxygen and harbors a tenth of Earth's species."

After template application, the model receives the following token sequence during training:

What the model learns to predict: During causal language modeling, the model must predict each token from all previous tokens. But we only care about its ability to predict the response tokens. The tokens up through and including the newline after ### Response: are treated as context (their losses are masked to -100). The model is penalized only for mispredicting the response tokens. After many such examples, the model internalizes: "when I see ### Response: after an instruction and some input, I should produce a concise summary of the input that addresses the instruction."

At inference time: You give exactly:

The model sees this partial sequence and generates the next tokens, which become the response. The final ### Response: header with no text after it is the model's cue to begin generating.

The key insight is that the template creates a deterministic boundary between "context to be processed" and "generation target." Everything before the response header is consumed as context. Everything after it is generated. The consistency of this boundary across all training examples is what makes the model reliable at inference time.