Instruction Format › Custom Chat Templates
Language AI HandbookAll chapters Instruction Format › Custom Chat TemplatesOne section · Sections in this chapter9 / 14 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Sometimes you need to define a custom template for models that do not have one, or modify an existing template for specific use cases. This might occur when you are working with a base model that was never instruction-tuned, when you want to add new roles beyond the standard three, or when you need to match a specific data format that differs from standard templates. Hugging Face tokenizers use Jinja2 templating syntax, which gives the flexibility to define arbitrary formatting logic.
Jinja2 is a Python templating engine commonly used for web development. In the context of chat templates, it allows you to write conditional logic, iterate over messages, and format content based on roles. The template receives the list of message dictionaries and produces a formatted string. Think of Jinja2 templates as a tiny programming language embedded in a string, giving you the expressive power to handle any conversation structure your use case requires.
# Load a base tokenizer without a chat template
base_tokenizer = AutoTokenizer.from_pretrained("gpt2")
# Define a custom chat template using Jinja2 syntax
custom_template = """{% for message in messages %}
{% if message['role'] == 'system' %}
[SYSTEM] {{ message['content'] }}
{% elif message['role'] == 'user' %}
[USER] {{ message['content'] }}
{% elif message['role'] == 'assistant' %}
[ASSISTANT] {{ message['content'] }}
{% endif %}
{% endfor %}
{% if add_generation_prompt %}
[ASSISTANT]{% endif %}"""
# Apply the custom template to the tokenizer
base_tokenizer.chat_template = custom_template
# Test it
test_messages = [
{"role": "system", "content": "Be helpful."},
{"role": "user", "content": "Hello!"},
]
custom_formatted = base_tokenizer.apply_chat_template(
test_messages, tokenize=False, add_generation_prompt=True
)Custom template output: [SYSTEM] Be helpful. [USER] Hello! [ASSISTANT]
The Jinja2 template iterates over messages, applies role-specific formatting, and optionally adds a generation prompt. This flexibility allows you to design templates that match your training data format exactly. The template syntax supports conditionals, loops, and variable interpolation, giving you full control over how conversations are rendered to text.
When creating custom templates, ensure your special tokens (like [SYSTEM], [USER], [ASSISTANT]) are added to the tokenizer's vocabulary as special tokens. Otherwise, they will be split into subword pieces, making them harder for the model to recognize as format boundaries. A [USER] token that gets tokenized as [, USER, ] gives the model three separate tokens to attend to instead of one, and the model must learn that this particular three-token sequence is always a structural marker, which is harder than learning a single unique token:
# Add special tokens to vocabulary
special_tokens = {
"additional_special_tokens": ["[SYSTEM]", "[USER]", "[ASSISTANT]"]
}
num_added = base_tokenizer.add_special_tokens(special_tokens)Added 3 special tokens [USER] token ID: 50258 [ASSISTANT] token ID: 50259
The output verifies that the new tokens are now part of the vocabulary with assigned IDs. Each special token receives its own unique ID. This keeps it will always be tokenized as a single token rather than being broken into pieces. After adding special tokens, you must resize the model's embedding layer to accommodate the new vocabulary entries. This is typically done with model.resize_token_embeddings(len(tokenizer)). The new token embeddings will be randomly initialized and will learn appropriate representations during fine-tuning. This initialization is an important detail: the initial random values should be drawn from the same distribution as the existing embeddings to avoid instability during early training steps.
Advanced Jinja2 Template Features
The Jinja2 syntax used by Hugging Face supports several useful features beyond simple conditionals. You can use {% set ns = namespace(found=false) %} to track state across iterations, letting you to handle patterns like "add a separator after every turn except the last." You can use filters like {{ message['content'] | trim }} to clean whitespace. You can use {% if loop.last %} to detect the final message and apply special formatting.
Real-world templates for models like Llama 3 use these features to handle edge cases: what to do when there is no system message, how to format the generation prompt differently for different contexts, and how to handle tool calls and tool results as special message types. Reading the actual Jinja2 template stored in a model's tokenizer_config.json is one of the best ways to deeply understand how that model's format works, because the template is the canonical specification of the format from the model authors' perspective.